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Summary 


Out-of-school time programs are well-positioned to enhance students’ Science, 
Technology, Engineering, and Math (STEM) skills, increase their interest in STEM fields, and 
increase their engagement in learning overall. In its 2011 Request for Proposals (RFP) for 
Out-of-School Time (OST) programs for youth in the elementary and middle grades, the New 
York City Department of Youth and Community Development (DYCD) encouraged programs to 
incorporate STEM subjects into their activities. In addition, DYCD’s RFP required OST 
programs to institute a system of continuous quality improvement (CQI), which includes an 
examination of the program’s goals and intended participant outcomes, observations and data 
collection, and adjustments to program practices based on this information. 


As part of its ongoing efforts to evaluate the implementation and effectiveness of the 
OST initiative, DYCD asked Policy Studies Associates, Inc. (PSA) to examine the ways in 
which OST programs implemented STEM activities, and to document their approaches to 
continuous quality improvement. In 2012-13, the first year in which STEM was part of DYCD’s 
expectations for OST programs, PSA visited five programs to learn about their experiences with 
designing and supporting STEM programming. Programs were selected through discussions 
with DYCD and preliminary interviews, and purposefully represented a variety of program 
structures and STEM implementation strategies. Three of these programs were center-based and 
two were school-based; two of the programs served participants in grades K-5, one served grades 
6-8, and two served both elementary- and middle-grades participants. This report is based on 
interviews with staff of these programs, as well as activity observations and document reviews. 


This report describes the strategies and approaches used by five OST programs to 
implement STEM activities. The first section of the report looks across the programs to examine 
the rationale for STEM implementation, and highlights areas where further technical assistance 
or guidance could strengthen implementation. The second section of the report contains a profile 
of each of the five programs visited for this study. The central findings of this study include: 


7 Programs implemented STEM activities using one of two approaches: integrating 
STEM themes into existing program activities, or creating stand-alone STEM 
activities. While all of the study programs sought to expose participants to STEM 
concepts, programs varied in the intensity of that exposure. 


7 Programs fit STEM activities into their existing program structures by either 
creating their own lessons or using existing curricula as resources from which to 
develop lessons; they did not adopt curricula wholesale. In some programs, 
STEM activity content was connected to or interwoven with program themes that 
were also infused through other activities. All of the programs visited for this 
study reported incorporating some type of hands-on learning into their STEM 
activities. 


a Programs recruited staff with varying levels of STEM experience. Based on 
observations at these five programs, staff with prior knowledge of STEM subjects 
and experience leading STEM activities were better able to design and implement 


inquiry-based activities that allowed participants to develop STEM process skills, 
such as prediction, measurement, and observation. Observations of staff without 
STEM instructional expertise indicated that staff may need additional assistance 
in implementing such activities. 


Programs did not use resources above and beyond their normal resources in order 
to implement STEM activities. They relied on their current funding, DYCD- 
sponsored trainings, and external partnerships to support STEM activities and 
staff development. At the same time, programs did not have sufficient funding to 
pay staff for the time they spent planning activities, or the time spent in 
professional development outside their programs. Programs need guidance on 
strategies to support lesson planning and to cover time spent in training, within 
their budgetary and time constraints. 


Program staff relied on existing networks to establish partnerships with external 
organizations in order to support STEM activities. While some programs could 
tap into the networks of staff members, in the first year of STEM programming, 
many struggled to identify partner organizations. Staff need additional guidance 
early in the process of planning for the program year to identify, develop, and 
sustain relationships with partners. 


Programs used various strategies to enhance school partnerships, including 
maintaining ongoing communication between OST and school-day staff and 
cross-staffing to support instruction. These partnerships supported the alignment 
of OST activities and in-school instruction and helped program staff address 
participants’ academic needs. 


While their methods for improving program quality varied, all programs had 
systems in place to plan and monitor instruction in their programs and provide 
feedback to staff, consistent with DYCD’s expectations for continuous quality 
improvement. However, STEM staff need additional support to develop effective 
activity plans that concretely define the activity’s learning goals, instructional 
strategies, and connections to past and future activities. For example, program 
directors and education specialists should review activity plans on a regular basis, 
observe activities and provide feedback, and facilitate ongoing reflection to help 
STEM staff improve their instructional strategies over time. 
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Introduction 


Since 2005, the New York City Department of Youth and Community Development 
(DYCD) has funded Out-of-School Time (OST) programs for youth that are intended to provide 
young people with high-quality programming that supports their social-emotional and cognitive 
development, at no cost to their families. Operated by nonprofit provider organizations, OST 
programs are housed in schools and community centers throughout New York City. 


To enhance the capacity of OST programs in helping participating youth build skills that 
will support their academic achievement, guidance established in the 2011 Request for Proposals 
(RFP)! issued by DYCD encouraged OST programs to offer Science, Technology, Engineering, 
and Math (STEM) enrichment activities. Elementary programs must provide at least two hours 
per week of either literacy or STEM-focused enrichment activities that are “structured and 
provide experiential learning opportunities, in addition to strengthening basic skills.” 
Middle-grades programs must offer a minimum of two hours per week of enrichment activities, 
“designed to build basic literacy and math skills as well as ‘21st century skills’ such as 
teamwork, problem-solving, and critical thinking.” Activities can focus on literacy, the arts, 
STEM, or a combination of these subject areas, and must offer experiential learning and 
skill-building opportunities. In addition, middle-grades programs must provide participants with 
opportunities to explore their interests. 


In addition, DYCD’s RFP required OST programs to institute a system of continuous 
quality improvement (CQJ) to “identify program strengths and weaknesses on an ongoing basis” 
and “raise quality through data-driven decision-making.” The recommended CQI efforts 
included an examination of the program’s goals and intended participant outcomes, observation 
of staff as they deliver activities, data collection on participants’ progress, and reflections upon 
and adjustments to program practices based on this information. 


As part of its ongoing efforts to evaluate the implementation and effectiveness of the 
OST initiative, DYCD asked Policy Studies Associates, Inc. (PSA) to document approaches to 
STEM activity implementation and continuous quality improvement occurring within OST 
programs. This report describes the strategies and approaches used by five OST programs. The 
first section of the report looks across the programs and examines how STEM activities were 
implemented, and highlights areas where further technical assistance or guidance could 
strengthen implementation. The second section of the report contains a profile of each of the 
five programs visited for this study. Each of these profiles describes the program’s structure, the 
content and instructional approach of its STEM activities, its alignment with schools, staffing 
strategies, and continuous quality improvement strategies. 


' Out-of-School Time Program for Elementary and Middle School Youth Request for Proposals: 
http://www.nyc.gov/html/dycd/html/resources/ostrfp.shtml 
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Study Methods 


In collaboration with DYCD, PSA identified five OST programs from which to gather 
information about STEM implementation and quality improvement efforts during the 2012-13 
school year, the first year in which STEM was part of DYCD’s expectations for OST programs. 
To select these programs, DYCD first identified 13 possible programs for inclusion, based on 
program proposals and DYCD knowledge of program implementation. PSA reviewed the 
proposals for these programs, and conducted brief phone interviews with the 13 program 
directors. The goal of these phone interviews was to learn more about their STEM activities and 
to gauge interest in participating in the evaluation. Following these interviews, PSA 
recommended seven of these programs to DYCD for further consideration. The recommended 
programs represented a range of implementation strategies, especially as related to use of 
curricula, school partnerships, and CQI. Through discussions with DYCD, five programs were 
selected for the study, including two school-based programs and three center-based programs. 
Two of the programs served participants in grades K-5, one served grades 6-8, and two served 
both elementary- and middle-grades participants.” 


Data collection took place in two phases. First, PSA conducted an initial set of in-depth 
telephone interviews with staff members from the five programs (January-February 2013) and 
reviewed documents that programs provided. Second, a team of PSA researchers conducted a 
one-day visit to each program in April or May 2013. Between the first and second phases of data 
collection, PSA prepared preliminary case study profiles for each of the programs and shared 
them with DYCD in order to identify emerging themes that were of greatest interest. While on 
site, PSA researchers observed STEM activities and conducted interviews with program 
directors, education specialists, STEM activity leaders, and (for school-based programs) school 
leaders. The research team drafted comprehensive site profiles following these visits; staff 
members from each of the programs were invited to review the profile of their program before its 
inclusion in this report. 


Per agreement with the New York City Department of Education’s Institutional Review 
Board (IRB), no names of schools or individuals are included in this report. Principals of school- 
based OST programs visited also approved the study prior to data collection, as required by the 
IRB. In addition, to further protect the confidentiality of the selected programs, PSA created 
pseudonyms rather than identify the nonprofit provider organizations operating the programs. 


Approaches to STEM Implementation 


There is growing consensus that out-of-school time programs are well-positioned to 
enhance participants’ STEM-related skills, increase their interest in STEM fields, and increase 
their engagement in learning overall. With the additional, informal learning time available in 
OST programs, participants have the opportunity to explore STEM topics through in-depth, 
hands-on activities (Traphagen, 2011). In addition, because of the population of youth they 


*One program had two DYCD contracts, one for elementary-grades and one for middle-grades; another program 
funded their middle-grades program through DYCD funds and funded fourth-and fifth-grade programming through 
other funding sources. 


serve, DYCD OST programs have the opportunity to spark an interest in STEM in girls and in 
youth from racial and ethnic groups that are underrepresented in STEM fields. By igniting 
students’ interest in STEM, OST programs like those funded by DYCD can increase the number 
of youth pursuing post-secondary education in STEM fields and pursuing STEM-related careers, 
and can also foster diversity within this STEM pipeline (Afterschool Alliance, 2010). 


Program Structure 


In response to DYCD’s expectations, OST staff at the programs in the study worked with 
each other and with their nonprofit provider organizations to determine how to implement STEM 
in ways that were aligned with both their existing program structure and with priorities for 
participant outcomes. Building on these established structures led to variations in STEM 
approaches, as described below. 


For some programs, STEM was a priority throughout the nonprofit provider 
organization. For two of the five programs in this study, the providers launched initiatives to 
promote STEM-related content in all of their programs (one provider did so before the release of 
the 2011 OST RFP, and the other did so after). Staff from both of these programs explained that 
their providers’ emphasis on STEM reflected national priorities for teaching 21* century skills 
(such as teamwork, problem-solving, and critical thinking) and for supporting in-school science 
instruction. The importance of STEM was communicated to programs through a number of 
channels, including in staff meetings for all provider-affiliated OST programs and in program- 
specific meetings. Through these initiatives, providers identified STEM curricula and training 
opportunities. The new STEM priority also influenced structural decisions at both the program 
and provider levels, including decisions related to staffing. 


Programs used two broad approaches to structure STEM activities. The OST programs 
in this study implemented STEM activities using one of two approaches: integrating STEM 
themes into existing program activities, or creating stand-alone STEM activities. Decisions 
about these approaches aligned with the overall vision of the programs and provider 
organizations, and also depended on whether STEM activities were intended for all students, or 
were part of a menu of enrichment offerings from which participants could choose. There was 
variation in how programs interpreted the expectations for STEM in OST. One program, for 
example, understood that they were required to include a weekly structured STEM block in the 
program, similar to activity blocks scheduled for other content areas. In contrast, another 
program viewed the new emphasis on STEM as an instructional approach, and developed 
strategies to integrate STEM themes into the program’s existing arts activities rather than 
scheduling STEM-specific activities. 


While all of the study programs sought to expose participants to STEM concepts, 
programs varied in the intensity of that exposure. Of the five programs visited, four engaged 
all program participants in STEM learning. One program allowed students to choose whether to 
participate in the dedicated STEM activity, but made efforts to weave STEM components 
throughout the program’s other activity offerings, such as learning digital photography in the 
yearbook club and applying mathematics concepts in music, so that all participants received 


some STEM instruction. One program offered a 30-minute science enrichment course twice a 
week for all participants as a way to support in-school test preparation. To give participants a 
meaningful choice about what they learn, the program also offered a longer, more intensive 
STEM activity that students could choose to sign up for; this activity was designed to excite and 
further engage participants with an established interest in science. 


Programs did not allocate greater resources to STEM relative to other program 
offerings. Rather, programs supported STEM implementation as part of general program 
planning. All of the programs visited paid staff hourly, with staff earning between $10 and $35 
per hour depending on their experience and qualifications in general; staff who instructed STEM 
activities were paid following the same standards as their colleagues instructing other activities. 


Preparation and planning time with colleagues are important elements to ensure the 
quality of activities. All five of the programs were able to compensate all program staff for some 
time spent in program staff meetings; three of the programs compensated staff for time spent in 
professional development. One additional program, because of budget constraints, could not pay 
staff their normal rate to attend professional development, but instead offered incentives such as 
comp time and gift cards. Again, program staff meetings and professional development 
resources applied equally to all program planning and improvement efforts, not just STEM. 

Only two of the programs were able to compensate staff fully for planning time. At another 
program, staff were paid one hour a month for staff meetings and an hour a month for lesson 
planning; however, the staff spent additional time planning their activities, without pay. Staff 
members at the other two programs also reported that they volunteered their time to plan lessons 
and research STEM activities. 


Without additional funding sources to implement STEM, programs often relied on 
donations from schools or used their creativity to secure materials for hands-on projects. STEM 
group leaders at one program explained that they often adjusted activities or found substitutes for 
materials to which their program did not have access. The program director stated, “We’ll just 
use what we have to make the experiment come to life.” Low-cost or free activities provided by 
partner organizations are also used to supplement STEM programming. For activities with an 
associated cost, one program director benefitted from a discretionary fund from the provider 
organization to cover costs. 


STEM Curricula and Instructional Strategies 


The five programs in this study developed their own STEM curricula or tailored existing 
materials to the teaching styles of staff, participants’ interests, program and provider priorities 
for outcomes, and the materials and space available to them. Some programs also chose to 
organize materials around program-wide themes and emphasized hands-on learning 
opportunities to support content delivery. 


Programs developed their own STEM curricula rather than implementing a packaged 
curriculum. Program staff felt that developing their own curricula or adapting existing materials 
allowed them to better meet program goals and respond to youth interests. For example, using 


knowledge of host schools’ curricula, staff in three of the study programs integrated concepts 
addressed during the school day as a way to support in-school instruction. Program staff also 
hoped to address participants’ interests to increase their engagement in STEM. At two programs, 
staff created their own activities, explaining that they were hesitant about using published 
curricula or activity guides because they were similar to activities participants had already done. 
Instead, staff developed their own activities, “using a variety of tools to boost the kids’ interests.” 

OST staff use a variety of strategies to differentiate in-school and afterschool STEM 
instruction. OST programs are uniquely situated to provide science instruction in such a way 
that complements but does not replicate in-school instruction. Indeed, several aspects of 
afterschool provide opportunities to learn science that supports academic success and 
engagement during the regular school day: increased opportunities for in-depth learning projects, 
informal learning environments that promote active participation, and flexibility for field trips 
and learning outside of the classroom (Afterschool Alliance, 2011). 


One program director explained that science instruction in the OST program is inherently 
different because her staff is not composed of teachers. She explained, “Their lessons [are 
delivered] in a more fun way.” While the director encouraged staff to align program content to 
learning standards, group leaders maintained autonomy to plan and implement activities that 
reflected their and the participants’ interests. For example, in addition to teaching through 
hands-on projects, the science group leaders often supported instruction by finding short videos 
to teach content in a way that drew in participants’ attention. At another program, staff viewed 
afterschool activities as a continuation of the school day and used the afterschool time to 
reinforce concepts learned during the school day through hands-on, participant-centered 
activities. A provider-level staff member explained, “If you’re familiar with teaching, you have 
your textbook and on the right-hand side of every page are these wonderful enrichment activities 
that support the lesson. You never have time to do them. [...] What we try to infuse [with the 
program staff is that] this is your fun time with the children.” 


Programs organized STEM activities around program-wide themes. In some programs, 
STEM activity content was connected to program themes that were also interwoven through 
other activities. One program, for example, connected STEM lessons to broad, program-wide 
month-long themes, such as dinosaurs and the universe. Program staff determined the focus of 
each theme, based on their own and participants’ interests, and integrated the theme into all 
program activities. Another program used theme-based arts projects and sought to infuse STEM 
topics and STEM -related skills into those projects. 


Programs differed in their approaches to developing activities that targeted STEM 
skills. Prior evaluations of the DYCD OST initiative found that programs that were effective in 
implementing skill-based programming communicated expectations for high-quality activities 
through intentional lesson planning and ongoing opportunities for staff reflection (Sinclair, B., 
Sanzone, J., Russell, C.A. & Reisner, E.R., 2012). All STEM activity staff in the programs 
visited for this study reported preparing for and researching their activities but varied in the 
extent to which they identified particular skills or knowledge targeted in each activity. For 
example, one program had established a strategic, intentional, and ongoing process for lesson 
planning. Staff developed lesson plans for six-week theme-based units that infused STEM 


concepts into arts programming, addressing both content themes and specific STEM skills and 
terms that students would learn about in each unit. Staff used paid meeting and planning time to 
discuss activity plans with other staff members, including the program director and education 
specialist, and to revise these plans to ensure that they were successful in helping participants 
develop the targeted skills. Although other programs also required staff to develop lesson plans, 
they tended to focus more on the content and age-appropriateness of the activity and less on 
identifying concrete outcomes. 


Hands-on, experiential learning opportunities supported content delivery. In the RFP 
for OST programs, DYCD stated that STEM enrichment activities should “provide experiential 
learning opportunities.” All of the programs visited for this study reported incorporating some 
type of hands-on learning activity into their STEM activities. One program, for example, used a 
two-part instructional approach: the first half hour was typically a discussion and Q and A 
between adults and participants, and during the second half hour participants engaged in 
independent or group hands-on work. At another program, STEM activities built from a 
lecture/discussion on day one, to an activity such as a worksheet or mini-lab on day two to 
solidify participants’ knowledge, to a more involved lab or activity on day three. 


A growing body of research on features of informal science learning suggests that 
informal science learning that includes hands-on, investigation-based activities and opportunities 
to link content to the real world may improve youth content knowledge, process skills (e.g., 
experimentation, observation, and analysis), and engagement in STEM-related fields (Minner, 
Level, & Century, 2010; Peterson, 2007; Tai, 2006). Science process skills were evident in some 
program activities observed for this study. For example, in one activity, participants learned 
about the concept of density and practiced predictive analysis to determine which of five nuts 
would be hardest to crack and, therefore, had the greatest density. Program staff interviewed at 
two of the programs emphasized that the hands-on activity components helped reinforce STEM 
concepts and supported learning outcomes. 


STEM Staff 


DYCD encourages programs to select staff with appropriate qualifications and skills for 
their roles and with at least two years of experience relevant to their position. In addition, 
DYCD expects programs to have an education specialist with at least a four-year college degree, 
experience in curriculum development, and, if possible, teaching credentials; in some cases, the 
program director may assume the responsibilities of the education specialist. DYCD envisioned 
that these staffing patterns would strengthen and support the quality of implementation of 
program activities, including STEM. The OST programs visited for the study employed adults 
with a range of professional backgrounds to lead STEM activities, including teachers and other 
adults with STEM expertise, as well as some without prior STEM expertise. All five programs 
had designated an education specialist. For two sites, the education specialist supported all of 
the provider organization’s OST programs; in only one program was the program director also 
the education specialist. 


Programs recruited staff with varying levels of STEM experience. One of the program 
elements that DYCD identified as supporting STEM activities is that staff should have interest in 
STEM or be experienced with STEM. Research on promising 4-H Science programs has shown 
that staff members and advisors in informal science programs with a background in science can 
provide a focus on inquiry and real-world applications of science, knowledge about science 
topics, and connections to potential partners and resources to the programs they lead (Riley & 
Butler, 2012). In two of the programs visited, all STEM activity leaders had prior experience 
teaching STEM subjects and/or strong knowledge of STEM content, including a science teacher 
and a college student majoring in biology. 


Although STEM experts can bring many benefits to informal science programs (Riley & 
Butler, 2012), with proper training and support, staff without STEM backgrounds can also lead 
activities (Donner & Wang, 2013). At one program in this study, the STEM activity leaders did 
not have any formal STEM content knowledge or training in STEM instruction. Rather, the 
program director emphasized that an ability to work with youth was an important criterion for 
hiring staff. To support STEM instruction, these staff relied on pre-made curricula such as 
Wonderwise and After-School Science PLUS, as well as the training offered through DYCD on 
implementing those curricula. Nonetheless, observations of these and other staff without STEM 
instructional expertise indicated that staff may need assistance in implementing inquiry-based 
activities that allow participants to develop STEM process skills. For example, an activity 
observed at this program, while hands-on, included few opportunities for inquiry-based learning. 
The group leader walked the students through the steps of building a circuit. On occasion, he 
took control and built parts of the circuits for youth rather than let them discover how to connect 
the pathways on their own. In a different program in which the STEM activity leader had 
knowledge of STEM subjects but little training in STEM instruction, PSA researchers observed a 
lecture-style session in which participants learned about cellular respiration and responded to 
basic questions about the concept as preparation for an in-program quiz. In contrast, in an 
activity led by an experienced science educator, youth learned about science concepts through 
group discussions and experiments in which they practiced predicting outcomes and analyzing 
results. 


Education specialists supported STEM instruction by providing feedback to staff, 
identifying curricula, and supporting instruction. Among the programs included in the study, 
education specialists had relevant qualifications to carry out their responsibilities: all of the 
education specialists had teaching expertise or experience working with youth in non-school 
settings. At the study sites, education specialists supported activity planning — sometimes 
exclusively for STEM and sometimes for all activities — and served as a resource for staff as they 
implemented their activity plans; they observed STEM instruction and gave staff feedback on 
ways to improve their teaching techniques. 


Two of the programs prioritized hiring an education specialist with STEM experience. For 
one program, staff from a partner school recommended that the program hire a science teacher 
from the school to serve as the education specialist. This education specialist focused only on 
STEM-related activities at the program. At another program, hiring an education specialist with 
STEM experience reflected the provider’s priority for STEM in programs. A staff member at this 
program explained, “I’m a licensed teacher and qualified to direct and develop appropriate 


afterschool curricula. However, STEM truly requires skills outside of the normal educational 
arena. [...] For hands-on, project-based STEM activities, staff must be comfortable with tools, 
applications, problem solving and have a broad understanding of components of STEM.” 


Programs participated in trainings provided by DYCD-affiliated technical assistance 
organizations or their provider organizations to support STEM and CQI. All programs visited 
for this study sent staff members — or education specialists and directors — to trainings related to 
STEM, including DYCD-sponsored training opportunities facilitated by The After School 
Corporation (TASC). TASC offered several opportunities for staff to participate in a “STEM 
101” session from October to February, which was designed to help staff learn about STEM 
practices, curricula, and how to implement STEM in programs. In addition, a series of “peer 
network” trainings offered in the spring were intended to help staff engage families in STEM 
activities, learn about the Next Generation Science Standards, build partnerships, and identify 
resources available to support STEM. Additionally, multi-session trainings in fall, winter, and 
spring focused on implementing specific STEM curricula (staff were required to commit to 
attending all three four-hour sessions for these curricula workshops). 


Staff from the study programs reported attending trainings on specific STEM curricula. 
For example, activity leaders from one program participated in trainings on Wonderwise and 
After-School Science PLUS curricula and found these trainings useful in planning and delivering 
activities to participants. Staff from another program also participated in After-School Science 
PLUS training. However, not all STEM program staff participated in all of the available 
DYCD-sponsored trainings. Also, some training opportunities (such as those that help programs 
build partnerships to support STEM) may have been more useful before the school year began. 
In addition, based on staff comments and activity observations, attending a training workshop 
does not guarantee that activity leaders will then have the capacity to implement curricula or 
practices addressed in the session. 


Ongoing training, perhaps delivered by education specialists, or through site-based 
trainings, could help STEM activity leaders better apply and implement approaches they learn. 
Some program staff did highlight the STEM trainings provided by their own organizations, 
program directors, or education specialists as particularly helpful. Programs hosted trainings for 
staff, usually during weekly or monthly staff meetings. At sites with only one or two STEM 
activity leaders, only those leaders attended STEM trainings; there were no program- or 
provider-wide STEM trainings that all staff attended. The two programs that incorporated 
STEM into their activity plans (either by infusing it into arts activities, or creating STEM 
activities for all students based on monthly themes) held in-house trainings specifically about 
how to do so. At these sites, all program staff were involved in the trainings, since all staff were 
expected to implement STEM. 


For one program, training focused more on the program director rather than on the staff 
who delivered activities, primarily because the provider hired teachers from host schools to lead 
its activities and these teachers already had extensive experience. The program directors from 
across this providers’ programs met weekly to discuss the provider’s emphasis on STEM, 
curriculum selection, and implementation. Program directors then shared that information with 
the activity leaders at their respective sites. 


Partnerships 


External partners can support STEM implementation in OST programs by drawing 
human resources, including networks of volunteers, and offering science expertise; often, 
services provided by external partners are available at a low cost to programs. All but one of the 
programs in this study used external partners to some degree to support programming. Most 
often, partner organizations provided mentors to programs to guide STEM implementation, 
volunteers to lead activities, and resources to support programming. 


Program staff used existing networks to establish partnerships with external 
organizations. Two of the programs included in the study benefited from having staff who were 
already connected to the STEM-related nonprofit network in New York City. In both cases, 
these staff members reached out to their contacts to coordinate supplemental activities for OST 
participants. For example, these opportunities included presentations by adults who work in 
STEM-related industries, nature hikes, and workshops for participants. The education specialist 
at one of the sites thought that programs without a staff member connected to these networks 
might struggle to find such organizations and build partnerships with them. 


Programs took advantage of DYCD-affiliated partners but would benefit from 
additional support to establish connections. Two of the programs applied for partnerships with 
the New York Academy of Sciences, an organization that has an established relationship with 
DYCD. Both programs were assigned a mentor who implemented two science units in the OST 
program during the school year. One program director felt that this partnership helped integrate 
high-quality hard science into activities. Some programs, however, struggled to establish 
partners. Time constraints to identify and contact partners may be a hindrance to programs. One 
program director, for example, explained, “I’m focused on programming. So, if [DYCD] brings 
to my attention somebody that wanted to partner with an afterschool program, then I’ll take the 
initiative.” Other program directors suggested that DYCD provide additional guidance to 
programs as they reach out to potential partners.” 


Alignment with Schools 


Research on OST programs has found that aligning OST program content with school- 
day curricula and instruction can support student learning and offer supports that complement 
and reinforce learning tasks that occur during the school day (Afterschool Alliance, 2011). The 
DYCD OST initiative aims to establish effective collaborations between OST providers and the 
schools which participants attend. The RFP for programs highlighted the importance of 
partnerships that support alignment between OST program content and school learning goals, 
calling for partnerships “founded on a common vision and involving collaborative planning, 
frequent and ongoing communication, and coordination” between OST programs and schools. 
All five of the programs in the study made some effort to support collaboration and coordination 


3 Although DYCD’s technical assistance provider (TASC) held one training on partnerships and the resources 
available to support STEM in afterschool, this training session was held in May (after most of this evaluation’s visits 
were completed). Program staff did not report reaching out to TASC for help in forming partnerships. 


between the OST program and in-school instruction, including maintaining ongoing 
communication with school staff and connecting activity content to learning standards. 


Program staff supported open communication with school-day teachers and 
administrators. At all five of the programs, the OST program made efforts to communicate with 
teachers and administrators of the schools attended by participants. Conversations with school- 
day staff addressed participants’ academic needs and topics addressed in participants’ classes. 
Programs used varying strategies to support communication between OST and school staff. One 
program, for example, partnered with its feeder school to institute an open-door policy, where 
OST staff were welcome to visit the classrooms of OST participants during the school day, and 
teachers were encouraged to visit the OST program. Another program had an established 
process for staff to communicate and coordinate with staff, including weekly check-ins that 
allowed staff to learn how participants are doing in their classes and areas in which they struggle. 


Programs sometimes used cross-staffing strategies to support OST and school-day 
content alignment. Two of the five provider organizations had deep roots in the communities 
served, having operated in their respective communities for 30 years or longer. In this time, the 
provider organizations had increased their visibility and gained recognition from community 
members, which, in turn, supported the credibility of the programs they operate. The two 
programs included in the study affiliated with these providers used these long-standing 
community relationships to involve school-day staff in OST activities. For one program, the 
provider’s relationship with the host school’s administrators helped in recruiting and hiring the 
program’s STEM instructor. For the second program, many former activity leaders — often 
college students studying education — went on to become teachers at the program’s feeder school 
and continued their support of the program. For both programs, involving school-day staff 
helped OST staff gain an understanding of participants’ academic needs and schools’ overall 
priorities for students. 


Through its relationship with the host school, one OST program’s staff worked alongside 
school-day teachers to help students during the school’s extended-day program. During the 
extended-day periods, OST staff went into classrooms and supported writing and math 
instruction. They provided individual assistance to all students, as needed, regardless of their 
enrollment in the OST program. One benefit of this approach was that program staff could 
observe teachers as they modeled instructional strategies. 


Continuous Quality Improvement 


In order improve the quality of activities in OST programs, DYCD expects programs to 
develop systems for monitoring and reflecting upon their work throughout the year. Although 
few programs could articulate a “Continuous Quality Improvement” or CQI plan, the programs 
in this study had indeed set up a range of systems to monitor instruction and student learning in 
their programs. 


All of the programs visited for this study used the same CQI methods to assess STEM 
activities and the staff that led them as they used for the rest of the program activities, although 
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one program reported exploring the use of a STEM-specific tool to supplement the evaluation 
tool currently in use (the NYSAN Program Quality Assessment tool). The CQI systems that 
programs set up had both formal and informal aspects. Formal aspects of CQI included weekly 
or monthly staff meetings, formal staff evaluations either once or twice a year, written reviews of 
lesson plans, and surveys of students and/or adults. Informally, directors reported observing 
programming on a regular (if not daily) basis, directors and staff reported conversing with 
students about their learning and the program in general, and staff reported giving one another 
feedback and support. 


Program staff used CQI systems to examine and improve programming, especially as 
related to professional development. Directors and education specialists sought to improve the 
activities that staff delivered by reviewing and giving feedback on lesson plans, observing 
activities and giving feedback on techniques to engage students, and responding to staff 
questions and challenges on an ongoing basis. All but one of the programs held weekly staff 
meetings (the other program met monthly). During these meetings, program staff reported 
discussing both program content and logistics. Staff at multiple programs cited their regular 
meetings as a strong aspect of their program that helped their team solve problems and helped 
them develop stronger activities. 


Program directors and education specialists also varied in the amount of review and 
oversight provided to staff as they developed activity plans. In one program, the review process 
included frequent rounds of director and education specialist feedback with staff revisions. At 
the other extreme, at the beginning of each semester the director of another program reviewed a 
list of topics to be covered on each day, but did not require or review lesson plans beyond that 
(although the activity leaders did spend unpaid time researching and preparing activities). 


CQI systems helped program staff monitor participants’ progress toward learning 
goals. Some programs set learning goals for their students that emphasized the development of 
21st Century skills. Programs also set goals for the content that they wanted students to 
understand at the end of a unit. Programs monitored student learning by using quizzes, surveys, 
pre-and post-tests, and informal conversations with students during activities. All of these tools 
were used to check student understanding of topics covered during the year; some programs also 
used surveys at the beginning of the year to uncover student interests and modify activities 
according to those interests. OST staff sometimes used program-specific tools to monitor 
progress. One program, for example, reported that students created portfolios for each activity 
unit, and that program staff met with each student at the end of the unit to discuss the student’s 
progress towards learning goals. In addition to checking understanding verbally at the beginning 
of each class, the staff also checked portfolios throughout the unit, and used it to determine 
whether the activity was successful in helping students meet these goals. 
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Summary of Findings and Recommendations 


DYCD’s focus on promoting STEM in OST programs reflects the current national focus 
on engaging youth in STEM in both formal and informal settings. Based on the five programs 
visited for this evaluation, in the first year of DYCD’s emphasis on STEM, OST programs took 
steps to implement STEM activities using strategies similar to those used for other subject areas. 
Programs also incorporated continuous quality improvement systems into their management 
structures, consistent with DYCD’s expectations. Findings from the experiences of these five 
programs are summarized below, followed by recommendations of ways to further strengthen 
efforts to improve program quality and impact, and STEM programming in particular. In 
general, patterns of implementation showed that: 


Programs did not use resources above and beyond their normal resources to 
implement STEM activities. They figured out ways to implement STEM 
activities within their current levels of funding, and by relying on 
DYCD-sponsored trainings and existing partnerships. 


Programs integrated STEM activities into their program philosophies and goals. 
They did not change their program to introduce STEM, but rather fit STEM 
activities into their existing program structures. In this vein, programs typically 
did not adopt outside curricula wholesale, but either used curricula as resources 
from which to develop lessons, or created their own lessons altogether. Programs 
often implemented theme-based and project-based STEM activities. 


Programs used the same continuous quality improvement methods for STEM as for 
other activities. Some programs used more formal tools for observing activities and 
providing feedback to staff than others, but all had systems in place for program 
directors and education specialists to monitor instruction and provide feedback to 
staff. 


Programs did not have sufficient funding to pay staff for all of the time they spent 
planning activities, or all of the time they spent in external training opportunities. 
However, programs did pay staff to attend staff meetings, which served as 
opportunities for coaching and troubleshooting. 


OST programs would benefit from the following supports to further strengthen their 
implementation of high-quality STEM learning opportunities: 


Offer guidance on resource allocation to support STEM. STEM activities 
require adequate planning, materials, and trained staff. However, program 
budgets and the part-time nature of most OST program staff positions are unlikely 
to change. Programs need more guidance on strategies to support ongoing, 
reflective lesson planning, within budgetary and time constraints. 


Facilitate participation in professional development activities. DYCD and its 
technical assistance providers offered trainings on STEM curricula. Although 
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staff reported that attending external trainings was helpful, programs noted 
challenges fitting trainings into staff schedules and paying staff to attend 
trainings. Helping programs identify funding sources to cover time spent at 
trainings could increase participation and build staff capacity. 


Provide additional guidance on STEM-related pedagogical techniques. 

Program staff were generally satisfied with trainings offered on curricula, but 
some expressed an interest in professional development focused on teaching 
higher-order science skills, without being tied to a specific curriculum. 
Additional, ongoing technical assistance addressing broader pedagogical 
techniques — such as developing inquiry-based activities — that engage participants 
and nurture their interests and curiosity would benefit both staff and participants. 
Offering this training to education specialists and program directors early in the 
year would enhance their capacity to better support staff throughout the year. 


Encourage programs to seek out leaders with STEM-related experience. Hiring an 
education specialist with specific expertise in STEM and an understanding of how 
STEM in OST can complement school-day STEM learning could help programs 
more successfully incorporate STEM. The education specialist could then provide 
support for strengthening instructional techniques to ensure that STEM activity 
leaders are incorporating best practices for informal science, such as inquiry and 
experiential learning. 


Help programs identify and approach partner organizations. While some 
programs tapped into the networks of staff members, others struggled to identify 
partner organizations that could support STEM programming. Programs need 
additional guidance early in the program year on strategies to identify, develop, 
and sustain relationships with partners. 


Promote an ongoing, reflective approach to activity planning. To support 
continuous improvement, STEM staff should not only be supported in writing 
activity plans, but program directors and educational specialists should review 
and discuss those plans with staff on a regular basis. This practice would help 
STEM instructors build their capacity to incorporate components of strong STEM 
activities into their plans, such as experimentation, observation, and analysis. 
Observation and feedback from program directors and education specialists could 
further help staff leading STEM activities improve their instructional strategies 
over time. 


13 


References 


Afterschool Alliance. (2010). Afterschool: Middle school and science, technology, 
engineering, and math (STEM) (Issue Brief No. 44). Washington, DC: MetLife 
Foundation, Author. 


Afterschool Alliance. (2011). Aligning afterschool with regular school day: The perfect 
complement (Issue Brief No. 50). Washington, DC: Author. 


Donner, J, & Wang, Y. (2013, Spring). Shifting expectations: Bringing STEM to scale through 
expanded learning systems. Afterschool Matters, 17, pp. 50-57. 


Institute for Learning Innovation. (2007). Evaluation of learning in informal learning 
environments. Paper prepared for the Committee on Science Education for Learning 
Science in Informal Environments. Retrieved from 
http://www7.nationalacademies.org/bose/Institute_for Learning Innovation _Commissio 
ned_Paper.pdf. 


Minner, D., Levy, A., & Century, J. (2010). Inquiry-based science instruction: What is it and 
does it matter: Results from a research synthesis years 1984 to 2002. Journal of 
Research in Science Teaching, 47(4), 474-496. 


National Research Council. (2000). Inquiry and the National Science Education Standards: A 
guide for teaching and learning. Washington, DC: The National Academies Press. 


Peterson, T., & Fix, S. (Eds.). (2007). Afterschool advantage: Powerful new learning 
opportunities. Moorestown, NJ: Foundations, Inc. 


Riley, D., & Butler, A. (2012). Priming the pipeline: Lessons from promising 4-H Science 
programs. Washington, DC: Policy Studies Associates. 


Sinclair, B., Sanzone, J., Russell, C.A., & Reisner, E.R. (2012). Approaches for integrating 
skill-based activities in out-of-school time programs. Washington, DC: Policy Studies 


Associates. 


Tai, R.T., Liu, C.Q., Maltese, A.V., & Fan, X.T. (2006. May 26). Planning early for careers in 
science. Science, 312 (5777), 1143-1144. 


Traphagen, K. (2011). Strengthening science education: The power of more time to deepen 
inquiry and engagement. Boston, MA: National Center on Time and Learning. 


14 


Program Profiles 


Amber Street Program 


Program Context 


Years in operation 30 

Grades served K-5 

Center- or school-based program Center-based 

Program operation Monday-Friday, 3:30 PM — 6:00 PM 
Number of participants 120 

Number of participants in STEM 120 

STEM enrollment All participants, grouped by grade 


STEM activity frequency/duration 1 hour/week for grades K-4, 2 hours/week for grade 5 


This center-based program has served its community for 30 years and currently provides 
OST programming to 120 participants enrolled in a local elementary school. Although the 
nonprofit organization operating the program has established a cultural and youth development 
focus for all its OST programs, program directors have flexibility in supervising and building 
staff capacity to meet the unique needs of each site. This program has a full-time on-site 
program director and is staffed by a team of 12 part-time OST staff members. Each month, 
teaching pairs for each grade-level cohort, composed of a head teacher and an assistant, develop 
theme-based lesson plans. The program’s schedule is structured so that different subjects are 
addressed each day: participants engage in literacy on Mondays and Thursdays, enrichment (e.g., 
dance, art, or music) on Tuesdays, STEM on Wednesdays, and physical activities on Fridays. 
Through STEM activities, staff hope to expose youth to science at an early age and help them 
understand how STEM subjects are relevant in “the real world.” 


The program has established a system of continuous quality improvement that includes 
professional development, monthly staff meetings, annual parent-staff meetings, student progress 
reports, and informal observations and feedback for individual staff members. The program has 
also implemented a series of workshops for staff to discuss ways in which STEM concepts can 
be integrated into existing program activities. 
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Approaches to STEM Programming 
School Alignment 
Connections between OST and school staff 


Although the program is located in a community center, it benefits from a close 
connection to its predominant feeder school. Before becoming a center-based program, the 
program was housed at the feeder school. Some program staff members were once teachers at 
the feeder school, and others have gone on to become teachers at the feeder school. As a result, 
program and school staff have established relationships and communicate regularly to address 
student needs. One program staff member commented, “Because the teachers are familiar with 
us, they will actually tell us, ‘This kid is not doing too well in math. This kid is not doing too 
good in science. This one needs help in English. Or they're just not doing homework.’” The 
program staff can then ensure that they address participants’ needs during program time; in one 
staff member’s words, “Because the thing is, we want them to do well. We don't want them to 
get left back or not do as well.” 


A teacher from the feeder school serves as the education specialist for the program, 
working five paid hours per week. In his dual role, he collects curricula from the teachers at the 
feeder school and shares these curricula with program staff in order to support lesson planning. 
He also meets with teachers and administrative staff at the school to discuss individual 
participants’ or grade-level needs and relays that information to program staff. The education 
specialist noted the importance of purposeful communication with the school’s teachers and 
principals. To facilitate communication, the program and the school recently instituted an open- 
door policy, such that program staff are welcome to visit and observe activities in participants’ 
classrooms during the school day, and school staff and teachers are encouraged to visit the OST 
program. One program staff member explained that she was encouraged to visit classrooms once 
or twice a month, but she went more often than that: “I used to go in every week. And actually, 
no, twice a week, even. It was my first time trying to speak to one of the teachers, so I thought 
we could build a relationship that way, and the kids would benefit from that.” 


The program director explained that at the beginning of the school year, the principal, 
assistant principal and school-day teachers receive a participant roster and a letter introducing the 
OST program and staff and welcoming them to visit the program. School-day teachers typically 
visit when the program holds special events such as performances, exhibitions, and fundraisers. 


Although neither OST nor school staff are paid for the time they spend visiting one 
another’s classrooms and activities, both the principal and the program director encourage their 
staff members to reach out to one another. The program director said, “I encouraged [the 
principal] to tell her teachers that we're not invading the classroom, but we are there to know 
what's happening in the daytime so that it makes a stronger connection.” The partnership with 
the school encourages relationship building among the staff and participants and supports STEM 
curriculum planning and additional support for participants in the STEM content areas. 


Using STEM to supplement school-day learning 


The staff member responsible for first-grade participants said she does not directly align 
the OST lessons to school-day lessons because participants become disinterested and say, “I’ve 
learned this already.” She tries not to repeat lessons, but instead approaches the material in a 
fresh way. For STEM, she aims to provide participants with new experiences, especially 
because first-graders do not receive science instruction during the regular school day. 


Program staff members in charge of older participants use the school-day curricula to 
develop lessons that supplement in-school lessons. The program director explained, “We hope 
to bring in more hands-on experiences for the kids, because in public schools they only have 
science once a week, or they have it once a week for one semester and then the next semester 
they have another extracurricular activity such as art or gym or something. So we want to let 
them know that science is still there — math, technology, and engineering. They only have 
computers once a week too sometimes. So we try to provide, in addition to what they're learning 
in school, something else so that they're more aware of their surroundings.” 


STEM Curricula and Instructional Strategies 


Participant-centered content 
; : uae . Approach to STEM curricula 
Teaching pairs lead activities for their 
grade cohorts every day, and are often supported ; ___ Physical and 
bya hich school volint The th meniat STEM topic area(s) biological sciences, 
y ahigh school volunteer. The three facilitators addressed Snoineering 
work together to develop and implement STEM 
activities that center around participants’ 
interests. All cohorts receive one hour of STEM 
programming a week, except for the fifth-grade 
cohort which engages in STEM activities twice a 
week. This cohort’s teaching pair includes a 
retired science teacher and a college engineering 
major. The program director explained that she scheduled more STEM time for this group in 
order to leverage the expertise of her program staff, and to enable students to create their own 
projects and learn more about math or engineering. 


Curricula used Developed by staff 


Staff, education 
specialist, program 
director, online 
resources 


Sources for activity 
plans 


The retired science teacher who works with fifth-grade participants tailors programming 
to prepare these youth for middle school: “Because I'm teaching the fifth-grade kids, they're 
going away to a junior high school next year. I was a junior high school teacher. One of the 
things that I always like to do is try to prep them for junior high school. So we do things that are 
similar to junior high school. We try to give them some basis.” 


Thematic lesson planning 


Program activities revolve around different themes each month; themes are thoughtfully 
selected according to participant and staff interests. Program staff connect activities to real- 
world concepts. The program director described, “Thematic instruction is the organization of a 
curriculum around macro-themes. Thematic instruction integrates basic disciplines like reading, 
math, and science with the exploration of a broad subject, such as communities, rain forests, 
river basins, the use of energy, and so on.” OST staff implement thematic activities for a full 
month, which allows a topic to come to life across various content areas. 


One staff member explained that she gathered ideas for her monthly themes by asking her 
participants at the beginning of the academic year what they wanted to learn. She said that 
participants often ask to learn about science-related themes. The first-graders, for example, 
wanted to learn about dinosaurs. In literacy, they read books about dinosaurs. In enrichment, 
they drew dinosaurs, and, on STEM days, they learned about fossils, dinosaur eating habits, and 
dinosaur life cycles. The fifth-grade cohort wanted to learn about animals, so staff members 
designed the monthly theme to explore exotic animals around the world. Students read about the 
habitats of animals on different continents, and each participant selected an animal to learn about 
for the month. Students learned to draw their animal and did research about it on computers. By 
the end of the month, students had created a world map of various exotic animals. Other themes 
chosen by participants included machines, gardening, and learning about the solar system. 


Developing in-house curricula 


In this program, the lead staff member for each teaching pair (the head teacher) is 
responsible for designing monthly lesson plans and receives input from their assistant as well as 
from the education specialist and program director. Staff members are paid for only one hour 
each month for lesson planning. 


For this monthly lesson planning process, the education specialist recommends that staff 
ask participants what they want to learn, and then choose what to teach based on both staff and 
participant interests. He explained, “If the teacher's not interested and the kid's not interested, 
then it's going to be very hard to teach a lesson. And I guess they put more value into what 
they're teaching and do a lot more planning and looking for ideas [when they are interested].” 
He recommends that staff work with him, research the topic on their own, or ask other program 
staff members for ideas about how to incorporate monthly STEM lessons related to their topic. 


The program director said that she seeks out STEM-related curricula trainings and 
workshops to obtain ideas for her staff, but often finds that many curricula are “very much what 
we have already done with the youth in our program.” For example, the staff found that the 
After-School Science PLUS curriculum included lessons similar to those they had already 
completed with participants. One staff member said she prefers to write her own lesson plans 
because published curricula may have been designed for a different grade level or require materials 
to which the program does not have easy access. She explained, “I don't like using lesson plans 
that are done before, like by other people. I like to, if anything, just restructure the whole thing. 

So if the basic idea is to introduce [youth] to a topic, let's say plants, and they have this whole 
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lesson plan and how to go through it, that's not what I want. I would try to use whatever resources 
I have to introduce a subject to [my cohort].” For example, one previous STEM activity required 
plants and flowers. She planned ahead by having the participants draw flowers during an art 
enrichment activity the week before and then used the flower drawings during the STEM plant 
activity the following week. 


Facilitating a workshop model 


STEM activities follow a workshop model. During the first half of each one-hour lesson, 
the staff member instructs the whole class, and during the second half, participants work 
independently, in pairs, or in groups to apply what they learned. This independent or group work 
is typically a hands-on activity. Staff members found 
that the workshop model encouraged participants to 
hold discussions, ask questions, and engage in Approach to staffing 
independent or group work. The workshop model fits Totaliaunbercl 
into the thematic structure of the program since staff 
activities are each designed to build participants’ skills 
and knowledge related to the theme. One staff member Eeucauon 

: ei specialist 
described activities in the workshop model as supports STEM 
interconnected, saying: “Each of them is stand-alone, 


but they are like smaller portions of a big thing.” NUPIBE I et Star 


who lead STEM 
activities 
During a first-grade STEM activity about ; 
simple machines, participants gathered on the open rug EFoIES= Ione! 
; development 
of the room for an overview about the STEM lesson Sources 
for the day. After a five-minute overview and a 
question- and-answer period, participants were split 
into two groups. Half went with the head teacher to External partners 
: supporting STEM 
gather around a computer in the back of the room, 
while the other half went with the assistant teacher 
who had a laptop. Participants watched a short 
animated video about simple machines on YouTube. 


NY Academy of 
Sciences; 
Cornell 
University; 
Lower East Side 
Ecology Center 


Salary range for $10.50-$12/hr. 


The large group gathered again and the head teacher STEM staff rv eon eae 
asked them to name and discuss the six machines einGlors 


described in the video. Participants then returned to 
their desks, copied definitions from an easel about 

simple machines, and drew three small pictures of a 
simple machine (e.g., a lever, a pulley, a wheel). They | Professional development 1 hr./month 
then completed a worksheet in which they matched Lesson planning time 1 hr./month 
images of machines to the concepts they had learned. 


Staff paid for: 
Meetings 1 hr./month 


Although a workshop model fits well into a weekly hour STEM activity period, it is not 
conducive to in-depth project-based activities. As one staff member put it, “I really want to do a 
hands-on activity with the kids [...]. But the thing is that sometimes — each day when we do a 
different category, like literacy, enrichment activity, STEM, and the physical activity, we have to 
drop everything and stop and then we have to come back like in the next two or three days. 
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Sometimes the kids will forget what we've done previously. And we have to recap everything 
what we did before they continue what they're doing.” 


STEM Staff 
Hiring and retaining qualified staff 


The program hires and retains educators with the experience and qualifications to develop 
lesson plans and skill-based activities. The average head teacher has worked at the program for 
seven years, and all six head teachers have or are currently working towards a master’s degree in 
education. The program director said she is fortunate to have highly qualified staff, and 
explained that she recruits through word of mouth. She asks her staff if they know of anyone 
interested in the position, and her only requirement is that they enjoy working with kids. 


The program also benefits from two long-term staff members with prior STEM-related 
experience. These staff members serve as the teaching pair for the fifth-grade cohort. Together, 
they develop STEM activity plans that address subjects such as building, engineering, and 
physical sciences. They also serve as a resource for the rest of the teaching staff in developing 
STEM activity ideas. 


All staff volunteer additional time to make the program successful, including observing 
and supporting instruction at the feeder school without pay, writing curricula each month, 
writing individual student progress reports, and communicating with parents outside of regular 
program hours. Although staff members receive an hour of pay each month to plan activities, 
one staff member pointed out, “But as anybody that teaches knows, an hour to plan a month and 
write up lesson plans is never enough. You're just telling me to go home, get it done at home. 
And that is not — I don't think it's fair.” 


Using STEM partnerships strategically 


The program has developed several partnerships that have provided staff with 
STEM-related experience and opportunities for field trips. With the encouragement of DYCD, 
the program applied to partner with the New York Academy of Sciences. A mentor from the 
academy was partnered with the program and teaches older participants (i.e., fourth- and 
fifth-graders) once per week for 10 weeks each semester. The STEM sessions provided by the 
New York Academy of Science are free, and all STEM-related materials are provided by the 
Academy. 


The program director encourages staff members to learn from the staff from partner 
organizations. She said, “[The Academy has] curricula online that they use. So I just told [staff], 
see if we can replicate it in our center without really having a mentor.” The program director 
prefers to use the science mentor for content areas her staff aren’t qualified to teach. She admitted 
that, despite having a science teacher and engineering major on staff, engineering and hard 
sciences are more challenging for the rest of her staff to teach. 
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In the summertime, staff from the Cornell University Nutrition Program come to the 
program to teach participants a series on nutrition. Last summer, this activity ran for about one 
week. In the past, the program has also partnered with the Lower East Side Ecology Center. 

The program director has found that activities provided by partner organizations work best in the 
summer or on full days of programming so that the program can make the most out of field trips 


or guest visits. 


Staff members also plan ahead and incorporate science guest presentations into the 
monthly themes. As one staff member explained, “Usually [the program director] is pretty good 
about letting me know ahead of time. She tells me this person will come in and do nutrition with 
you; would you be interested in that? [If] I say yes, I try to find a way to tie connections to the 
person who comes to do nutrition with the theme that we're doing.” 


Continuous Quality Improvement (CQI) 


The program’s CQI process includes the use of 
staff training tracking cards, monthly staff meetings, 
annual student progress reports, parent-staff meetings, 
and monthly informal observations and feedback by 
the program director and education specialist. 


Staff members rely on one another for 
observations and feedback. One staff member worked 
closely with her assistant teacher for instructional and 
implementation feedback. She said, “I get feedback 
from her. I ask her what she thinks about my lesson, if 
it goes well or doesn't go as well, or maybe how we 
should change it a little bit. [...] I also give her my 
feedback, what I think she could do better.” Program 
staff are required to attend 30 hours of professional 
training every two years, and document hours. 


Participant-parent-staff annual meetings 


The program has created templates of student 


Approach to CQI 


Staff submit 
lesson plans 
for review 


Frequency of Monthly 
staff meetings 


Program Student progress 
Assessment reports, 
tools used Student/parent/staff 
meetings, Staff 

evaluation assessment 


Frequency of 
staff 
evaluation 


Annual; additional if 
necessary. 


Assess student 

Use of progress and provide 
feedback feedback to students, 
Refine program 

activities/implementation 


progress reports that staff use to summarize student outcomes. Staff use student progress reports 
every spring during parent-staff meetings. The progress report assesses a student in six 
developmental areas that align with the program’s goals of focusing on the development of the 
whole child: (1) academic level/cognitive development; (2) social development; (3) physical 
development; (4) emotional development; (5) attitudes and behavior; and (6) work habits and 
class participation. Staff review the progress report with the participant and his or her parent. 
After the participant and parent create a goal for the upcoming year in one of the six 
developmental areas, the document is signed and then reviewed the following year. The 
conferences are scheduled during programming hours, so that one staff member supervises the 
class while the other meets with the parents. The program director explained, ““We hope that 


A-7 


with the combination from both [this program meetings and regular parent-teacher meetings at 
the school], parents will gain a better understanding of their children.” Discussing participants’ 
academic and social development with an afterschool staff member may give parents additional 
information about their children’s’ progress. 


One unexpected outcome of the parent-staff meetings is that the program staff now 
facilitate communication between the feeder school and participants’ families. When asked what 
the feeder school valued most from the OST program, the program director said it was their 
communications with parents. Holding parent-staff conferences has also allowed program staff 
to learn more about the families they are serving and to better adjust programming to meet 
participants’ needs. For example, one lead staff member said that many families did not have the 
resources to provide STEM-related activities at home, such as taking their children to the 
museum or enrolling them in a science camp. Instead, participants can enjoy such experiences 
through the program. 


Informal observations and discussions 


Staff use informal approaches to assess the program’s effect. As one program staff 
member explained, “I hope that they understand the concepts that we're trying to teach them. 
That's what the discussions are for. So we try to talk to them as much as we can about what we 
did, instead of just telling them this is what we're going to do and that's it. Because they need to 
hear themselves or hear each other, what they all think. That's the learning experience.” Staff 
members also bring content and questions from participants to the monthly staff meetings in 
which they receive support from each other related to implementation, activity content, and 
individual student concerns. 


One head teacher uses the Know/Want to Know/Learned (KWL) Chart, a common 
teaching strategy designed to support the learning process. She explained how she uses the tool 
each month: “In the beginning we started with a KWL chart, what they know about dinosaurs. I 
want the kids to tell me what they already know about it and then we make a list. And then after 
the K chart, we did a W chart. They asked me questions and they were curious and wanted to 
know what dinosaurs are about. So we're making two different charts. And then the last chart 
[is the] learning chart. After we've studied all about dinosaurs, understand what they're about, 
the kids will tell me what they know — it's like an assessment of what they know about it, and 
then we have the kids write about it or maybe journal about the things they know about 
dinosaurs.” 


By using these approaches to support STEM implementation, the program engages 
participants in STEM learning opportunities. The education specialist said, “We make it fun 
here. It’s not a place where there's so much pressure. The kids are here to learn, that's the main 
goal, to teach them as much as we can and teach them things that they wouldn't learn at school or 
at home, but open it up for them to ask and feel free to do whatever they need to do to learn 
something.” 


Arts Infusion Program 


Program Context 
Years in operation 5 
Grades served K-5 
Center- or school-based program School-based 
Program operation Monday-Friday, 2:30 PM — 6:00 PM 
Number of participants 120 
Number of participants in STEM 120 
STEM enrollment All participants, grouped by grade 


STEM activity frequency/duration —_‘!nfused throughout activities; 
Two six-week rotations per year focus on STEM 


Overview 


The program, located in a K-5 elementary school, serves approximately 120 participants, 
including some from a neighborhood school. In this program, experienced Teaching Artists 
develop project-based activities throughout the year that are implemented in six-week rotations, 
and two of these activities must be STEM-specific. In STEM-infused arts activities, participants 
have sketched the human brain, connected movement and geometry by making movement maps 
for a dance recital, and learned fractions through music composition. A director and education 
specialist from the nonprofit provider organization supervise and build staff capacity for all of 
the organization’s youth programs. In addition, each program has a full-time program director 
and is staffed by a team of staff members who each have specific functions in implementing and 
supporting program activities. 


This program’s provider organization focuses on arts and literacy in all its OST 
programs. In responding to the most recent DYCD RFP, the provider organization worked with 
all program staff to identify the extent to which STEM concepts were already integrated into the 
arts-based activities, and to develop strategies to make STEM programming more intentional. 
Through the ongoing continuous quality improvement process, the education specialist and 
program directors continue to build staff capacity to implement STEM programming. 
Homework Warriors assist participants with their homework for an hour a day and coordinate 
closely with school-day teachers to ensure that participants’ learning needs are met. Program 
Apprentices, who are high school or college students, support the Teaching Artists and 
Homework Warriors in activity implementation. The program has also established a multi- 
faceted system of continuous quality improvement that includes ongoing staff training, staff 
meetings, a lesson planning and review process, an assessment of student progress at the end of 
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each project unit, parents and teacher surveys, and observations of and feedback to individual 
staff members. 


Approaches to STEM Programming 
Program Structure 
Theme-based programming 


The program integrates STEM concepts through theme-based arts activities that are 
designed to help participants develop inquiry and analysis skills, consistent with the provider 
organization’s model for all of its OST programs. To support this integration and respond to the 
DYCD expectation for STEM programming, the education specialist trained staff at the 
beginning of the year to identify ways in which they were already doing STEM in their arts- 
based activities. The education specialist explained, “Because if you can acknowledge that 
you're already doing something then it's easier to build on it. So how do we now make this more 
detailed, more highlighted or a bigger part of what we're doing?” One staff member noted, 
“When we first had the workshop about STEM, I was confused, like what is this, it's going to be 
so complicated. But then I realized we used this all along. [STEM] was always in our lesson 
plan, but we hadn't noticed. [...] So it wasn't that hard to incorporate it, because we were already 
doing it, but weren’t aware of it. Science is in a lot of our little projects and activities that you do 
when you build stuff, and math is everywhere.” 


According to the education specialist, by incorporating STEM into activities, the program 
aims for STEM to become a “relatable experience” for participants by not only teaching them 
foundational skills to apply STEM concepts, but also giving them opportunities to practice these 
skills. For example, in a six-week thematic unit on superheroes, participants researched social 
issues that affected them. After they discussed these issues, they then created superheroes to 
combat their chosen social issue in a nonviolent way. Participants created posters about these 
superheroes while also analyzing different poster styles. According to the Teaching Artist who 
developed this activity, participants developed STEM skills throughout the process. “Social 
sciences, that's bringing STEM into it. With the research component, writing and typing is 
bringing technology into it. Engineering and math come into play when it comes to 
composition.” The arts component of the activity incorporated geometry in understanding 
dimensions and space in figure drawing. The activity also incorporated the study of anatomy in 
having to draw a human body for the superhero. 
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STEM Curricula and Instructional Strategies 


Program staff develop their own lesson plans that 
infuse STEM concepts. With support from the education 
specialist, staff use various resources to plan activities. 
Staff may build on prior experience, or search for new 
ideas online (for example, through the Incredible Art 
Department or PBS’s Arts 21). 


Arts-focused instruction 
The school principal values the support that the 


program provides to participants through the arts-based 
opportunities that supplement the school day. The 


Approach to STEM curricula 


General 
science, Social 
science, 
Physical science 


STEM topic 
area(s) 
addressed 


STEAM 
activities 
developed by 
staff 


Curricula used 


Education 
Sources for specialist, 
activity plans Online 
resources 


principal believes that the program should support the school through activities that the 
participants do not receive during the school day: “I believe that it should probably be a lot of 
extracurricular activities to make the children more at ease, because we’re not able to provide 


that during the school day.” To facilitate the alignment 
between these enrichment activities and school-day 
learning, information about the school-day curriculum is 
made available to program staff, and the calendar of 
units of studies is also available to the program for each 
month. In addition, by sitting on the School Leadership 
Team and being in frequent contact with the principal 
and with the teachers, the program director can relay 
information about the school-day curriculum to program 
staff. 


STEM Staff 


The program is staffed by 13 individuals, 
including three part-time Teaching Artists, three part- 
time Homework Warriors, six volunteer program 
apprentices, and one full-time program director. Each 
staff member plays a specific role in the program: 


rT Teaching Artists are expected to “develop 
and facilitate a progressive, innovative 
arts-and literacy-based curriculum” 
(according to their job description). 
These staff members must possess a 
BA/BFA and have a minimum of four 
years of experience teaching art. 
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Approach to staffing 


Total number of 
staff 


Education 
specialist 
supports STEM 


Number of staff 
who lead STEM 
activities 


Professional 
development 
sources 


External partners 
supporting STEM 


Salary range for 
STEM staff 
HW Warriors 
Teaching Artists 


$10/hour 
$20/hour 


Staff paid for: 


Meetings 


Professional 
development 


Lesson planning time 


7 Homework Warriors are responsible for running two homework periods each day. 
These staff members must be enrolled in college and have two years of 
experience working in an afterschool program. 


7 Program Apprentices are high school or college student volunteers who work with 
a grade-level group throughout the program activities, and support the Homework 
Warriors and Teaching Artists. 


Staff members work together to support participants and program activities. At the start of 
the program, staff and participants meet in the cafeteria for a physical activity. Afterwards, half of 
the participants stay in the cafeteria to work on homework with the Homework Warriors, while the 
other half participates in activities with the Teaching Artists. Later, participants rotate. 


Although staff members have previously attended trainings offered through DYCD’s 
technical assistance providers, most STEM training is done in-house, either by the program 
director or the education specialist during the weekly whole-staff meetings. The education 
specialist explained that in a recent STEM training, discussion focused on the rationale for 
integrating STEM into the program. “It had a lot to do with the global economy and where we 
are with education and the global scale. It's not just about education in the inner city, right? It's 
about our economic future, if you will.” She said getting staff to understand the reasoning 
behind the STEM initiative was vital to how they will implement STEM activities. She added 
that reviewing lesson plans, working across disciplines, and using STEM vocabulary has further 
supported intentional implementation of interdisciplinary STEAM (Science, Technology, 
Engineering, Arts, and Math) activities. “We do a lot of interdisciplinary work. A dance teacher 
might go to a visual artist and say, “Hey, I'm working this dance choreography piece, and we're 
talking about geometry.’ One teacher drew movement maps for a piece of choreography. So 
then you have this sort of linear plotting in space, and then the kids had to choose where their 
plot was and their paths couldn't cross at a certain time to avoid collision. So you have them 
using math to figure out how they can move through space efficiently.” 


Alignment with Schools 


The Homework Warriors support participant success in STEM by helping with 
mathematics homework. The program has established mechanisms for staff to communicate and 
coordinate with school-day staff to ensure that they are knowledgeable about school-day learning 
and instructional techniques. First, the program uses a push-in model, in which the Homework 
Warriors assist school-day teachers in an extended day classroom three days a week. Second, 
Homework Warriors conduct weekly check-ins with school-day teachers about the needs of 
students, and finally, program leaders have established open communication with school 
administrators and teachers. 
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Pushing into the school day 


Some program staff work alongside school-day teachers in supporting students during the 
school’s extended-day program. These extended-day periods are intended to help students who 
are struggling in school, particularly in preparation for the state assessments. In this push-in 
model, Homework Warriors go to the classroom and help the school-day teachers as they 
provide additional instruction in writing and math. They help ensure that every child is on the 
same page and understands the material by providing individual assistance as needed. Through 
this approach, Homework Warriors can observe teachers as they model instructional strategies. 


Communication with school-day staff 


In addition, all Homework Warriors are expected to talk to teachers on a weekly basis 
about participants’ school-day performance, introduce themselves at the beginning of the year, 
and schedule a regular check-in time. As that relationship develops, the teachers also approach 
the staff informally. The education specialist noted, “What’s so great about the check-ins is that 
there is a relationship formed. [...] Once the teacher knows that we’re actually serious about 
helping the kids, then they’re more interested, and they’re like, ‘I noticed this is going on, can 
you help with [this or that].” The program director noted, “It’s important to come in and meet 
with those teachers and say, ‘This is what we're doing, this is how we incorporate literacy and all 
these core curriculums that need to happen and this is how we do it.” 


These check-ins provide opportunities for the program staff to find out how participants 
are doing in school and identify the subjects participants are struggling with. The education 
specialist noted, “Recently, the fifth-graders took a math pretest and a lot of fractions were 
mentioned and problem words. So now we know we should be focusing on these things to build 
those skills and make them ready for the state test.”” One Homework Warrior commented, “I 
meet with [teachers] about once a week to talk to them about the children, like what does this 
child need help in, how is he behaving in class, what problems are there?” The Homework 
Warriors review participants’ needs and progress with the participants themselves. For example, 
a Homework Warrior described what she might say to participants: “The last time I talked to the 
teacher, you weren't completing homework [...] I talked to the teacher this time, she says you're 
doing wonderful, your math is improving and you're doing your homework.” In order to 
communicate with the teachers of participants from a neighboring school (not the main host 
school), program staff ask parents to send notes to these teachers, or include notes in students’ 
school folders. Staff members share what they learn about participants with one other during 
their weekly meetings. 
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Continuous Quality Improvement (CQI) 


Approach to CQI 
Ongoing lesson plannin 
eorn P 6 Staff submit 
: . lesson plans for 
The program has established a strategic and review 
intentional process for lesson planning to ensure that ‘ 
hoa ‘ : ; : requency o 
activities are aligned with program’s goals. In this staff meetings Weekly 
process, staff: 
Student 
: Brainstorm possible thematic ideas, and Assessment Pamuouos .F afent 
: tools used and teacher 
discuss them as a group surveys, Review 
of student report 
rT Identify specific vocabulary terms gaids 
participants will learn and outcomes they Frequency of Gace nsnwsek 
will achieve through each activity staff evaluation 
. . Assess student 
| Develop a weekly lesson plan, which is progress and 
then reviewed by the program director Use of feedback _ provide feedback 
: oT; to students, 
and education specialist Rietine progam 
activities 
= Discuss lesson plans during weekly staff 
meetings 
a Receive individual feedback on lesson plans 


Teaching Artists work closely with the education specialist, the program director, and 
each other to plan and refine their lessons for each six-week thematic unit. First, staff members 
submit an idea for a theme to the entire program staff during a weekly staff meeting. These ideas 
are often inspired by staff members’ own artistic expertise or background, adapted to be age- 
appropriate and reflect the developmental needs of participants. Staff receive feedback from 
their colleagues on how to develop that theme throughout the six-week interval. During these 
feedback sessions, a Teaching Artist reported that staff are “looking at our lesson, looking at the 
wording, discussing the activities, any concerns that we may have individually as Teaching 
Artists, and collectively coming together to figure out a better solution or another alternative.” 
Teaching Artists then write weekly lesson plans that are reviewed by the program director and 
the education specialist, who provide individual feedback to all the Teaching Artists involved. 
Each week the lesson plans are refined to meet the needs of participants. The education 
specialist described the lesson planning as “ongoing” and a result of “a lot of group discussion.” 


One Teaching Artist said, “Lesson planning for me is coming up with a project. I think 
of the big idea first and then I work my way backwards. That's how I was taught to lesson plan: 
Come up with the end result, the end product, and then work your way to figure out the steps that 
lead up to that.” This Teaching Artist continued, “I think about how I’m going to incorporate 
technology and literacy, and then figure out the activity or the many activities that will give 
participants the skills that it will take to do the final project.” The final projects are presented at 
the end of the lesson series on family nights, during which participants showcase their work. 
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These projects might be a choreographed dance, an artistic piece, or other form of artistic 
expression. As compensation for this extensive planning, program staff are paid for one hour a 
week of planning and for a two-hour staff meeting. In total, staff are paid 21 hours a week. 


Monitoring student learning 


For each theme rotation, program staff identify the essential vocabulary and skills that 
students are expected to learn by the end of the rotation. For example, one program rotation 
focused on animals and incorporated concepts of animal anatomy, movement, and skeletal 
structures. As one staff member explained, vocabulary was “anatomy-specific and then maybe 
science-specific about different animals, habitats, environments, things like that.” Throughout 
the rotations, staff introduce vocabulary in various ways. For example, the dance instructor 
might develop an exercise in which students physically demonstrate the meaning of the word, in 
order to help them retain the information. 


The program’s CQI process includes reviews of participant portfolios, which contain 
works of art accompanied by a standard portfolio review document (completed for each program 
rotation). At the end of each program rotation, Teaching Artists and participants meet one-on- 
one to examine participants’ portfolios. The portfolio review is aligned with the vocabulary 
words and the skill development goals that staff wrote into lesson plans. Program staff identify 
areas in which participants have successfully developed their skills, along with areas in which 
lesson planning should be strengthened in order to have a greater impact on students. According 
to the program director, through these portfolio reviews, staff can “see certain patterns and 
certain words that children couldn't retain, they can reflect on their own lesson, like maybe I 
didn't focus on it long enough or maybe they needed to try something different.” Teaching 
Artists review portfolios with individual participants for about five minutes each. They ask 
participants what they think they learned in order to better understand if participants are 
understanding the lessons. According to a Teaching Artist, these portfolio reviews “hold 
[students] accountable, but also provides them with feedback. [...] It gives them a sense of 
responsibility that they probably don’t realize that they have.” 


Although program staff use portfolio reviews to determine if they were successful in 
meeting the goals of their activities, the Teaching Artists also described a more ongoing, 
informal system of assessing progress. One Teaching Artist said, “You never wait until the end, 
you're always assessing throughout.” Staff review concepts in the beginning of each class to 
assess whether participants remember previous lessons and if they understood the main point of 
the lesson. The program also measures program effectiveness through annual end-of-year parent 
and teacher surveys. Parents are asked to describe their child’s improvement in school subject 
areas, homework habits, and social behavior. Teachers are asked about participant successes, 
what they value most about the program, and for suggestions to improve communication with 
program staff. Both surveys also offer parents and teachers the opportunity to provide feedback 
about program content areas that can be improved. Starting next year, the program director 
reported that the program will survey parents about their children’s academic and social skills 
before the program as well as after the program in order to detect potential changes in students’ 
knowledge and behaviors. 
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Ed Connections 


Program Context 


Years in operation 8 

Grades served 6-8 

Center- or school-based program School-based 

Program operation Monday-Thursday, 2:00-4:00 PM 
Number of participants 100 

Number of participants in STEM 20-25 each three-month cycle 
STEM enrollment By interest 


STEM activity frequency/duration Three hours per week 


Overview 


In response to both the new DYCD priorities for OST programs and to the national focus 
on science education, the nonprofit provider organization operating this program initiated an 
emphasis on STEM-related content in all of its OST programs. The goals of the organization’s 
STEM initiative are to increase participants’ engagement and interest in science-related content 
and to encourage participants to develop problem solving and critical thinking skills. 


This school-based OST program serves approximately 100 participants in grades 6-8. 
The program, which has operated for eight years, provides a mix of academic and recreation 
activities to participants. Participants explore their communities and local environments through 
field experiences, project-based activities, and independent research centered on environmental 
science and physics. A full-time program director, supported by staff from the provider 
organization and the agency’s education specialist, oversees all program activities. 


Participants engage in activities in three-month cycles. At the start of each cycle, 
participants choose the activities in which they would like to participate; they may rejoin the 
same activity for the next cycle. The science activity, which meets three hours each week, draws 
approximately 20 participants each cycle. While many of the participants who engage in the 
science activity re-enroll, staff said that they also encourage participants who might shy away 
from science activities to try them. One staff member said that the goal is to have all participants 
take science during at least one of the activity cycles. The program director does considerable 
outreach to participants who do not engage in science activities, encouraging them to attend the 
science group if they have not yet done so during the year. 


The science activity addresses a range of STEM content, mostly centering on 
environmental science and physics. Staff members provide hands-on learning experiences to 
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support learning outcomes. Past activities have included making paper airplanes to demonstrate 
principles of aerodynamics and building simple bridges during an engineering lesson. In 
addition to the science activity, program staff incorporate STEM components into other activities 
(e.g., digital photography in the program’s yearbook club, computer software in the newspaper 
club, mathematics in the music club) so that all program participants receive some type of 
STEM-related instruction while not participating in the science activity. 


On the day visited, participants completed a “Hard As a Nut” activity to explore the 
concept of density. After a brief overview led by the activity leader, participants practiced 
predictive analysis to determine which of five types of nuts would be hardest to crack and, 
therefore, possessed the greatest density. In groups of three or four, participants developed 
hypotheses about each nut’s density and the number of pennies required to crack each nut. Then, 
using the procedure proposed in the activity guide, they tested their hypotheses. Each group 
designated a member to record observations and to present their findings to the entire class. 


For all OST programs, the provider organization prioritizes hiring licensed teachers to 
lead activities, assisted by college students pursuing degrees in education or related areas. At 
this program, a science teacher employed by the host school leads STEM activities. According 
to the program’s leadership, hiring a teacher who is knowledgeable and enthusiastic about 
science-related content has increased participants’ excitement and engagement. Additionally, the 
program benefits from a strong relationship with its host school. Teachers and administrators 
regard the afterschool program as an extension of the school day, and lines of communication 
between OST program staff and faculty were described as a “cornerstone” of the program. 


Approaches to STEM Programming 
Program Structure 
Nonprofit provider organization emphasis on STEM in OST programs 


The provider organization has promoted an increased emphasis on STEM in all its OST 
programs. Although the provider organization had previously integrated science and math into 
many of its programs, the new emphasis was designed to bring STEM to the forefront and 
bolster their STEM efforts, especially as they relate to engineering and technology. At the 
provider level, staff now meet weekly to discuss STEM curricula selection and implementation 
at programs. 


At school-based OST programs such as this one, the focus on STEM has been a natural 
fit for the provider. One staff member said that principals embrace the concept of STEM in 
afterschool programming as a strategy to help participants become critical thinkers and increase 
their confidence and self-efficacy related to science. She explained, ““We want them to embrace 
[science], and it’s only going to happen if they can practice with it and [...] if there’s that 
exposure.” Principals have told program staff that integrating STEM content into the afterschool 
program has been beneficial because it has allowed participants to explore content in ways that 
they cannot during the school day due to the emphasis on standardized tests. 
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The provider organization’s emphasis on STEM, as well as the host schools’ priorities, is 
communicated through a number of channels. Program directors meet with provider-level staff 
at weekly staff meetings during which they discuss STEM implementation at each program. The 
weekly meetings allow program directors to share best practices and challenges as they relate to 
STEM activities. Although the program directors and provider-level staff work closely together, 
the program directors often take the initiative in deciding which aspects of science should be 
prioritized in their programs. At their respective programs, program directors share the 
information discussed with the front-line staff who carry out activities. At this program, OST 
science activities are led by a teacher from the host school. She is supported by the education 
specialist and the program director, who often provide curricular resources and materials to 
support instruction. 


The programs also rely on a train-the-trainer model to transfer professional development 
information to the front-line staff. Related to STEM, the education specialist and program 
director have attended trainings organized by The After-School Corporation (TASC) and the 
Partnership of Afterschool Education (PASE). The education specialist also attended a training 
provided by the Youth Development Institute (YDI) related to Common Core; the education 
specialist mentioned that he has used this particular training to work with activity leaders across 
all the programs to ensure that, as they develop lesson plans, the content is aligned to the new 
standards. 


STEM Curricula and Pedagogy 


Approach to STEM Curricula 


Engaging older participants in STEM 


Program staff agreed that an initial challenge lenin Sener Pte re 
was developing high-quality STEM activities that 
would appeal to middle school-aged participants. Mix of published 
Program staff discussed three strategies to maintain the | Curricula used Design it and 
interest of this age group: hiring knowledgeable staff, activities developed 
providing leadership opportunities for participants, and by staff 
offering participant-centered content. Hiring staff with Piovider 
STEM expertise is especially important for programs Curricula source(s) —_ organization, Online 
that serve older youth. One staff member said, “These resources, Self- 


; : ; I icul 
kids are at this age where they know [...] if someone ee ait 


doesn't know what they're talking about and they can 
sense that, and, as soon as they sense that, they start to 
drift away.” The program’s leadership and staff believe that hiring knowledgeable and 
enthusiastic staff has increased participant engagement in science and brought in those who 
otherwise may have not participated in STEM-related activities. 


Program staff engage participants by giving them more ownership of the program: “You 
have to allow them to take on responsibility and feel like they are really important.” One 
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participant, for example, heads the garden committee that maintains the school’s outdoor garden. 
Her responsibilities include collecting materials for the compost bin. 


To plan activities, the teacher who leads science activities explained that she often asks 
participants — many of whom are in her school-day classes — what they would like to learn. The 
teacher said that this approach has increased participants’ engagement in science, especially 
among those who were considered “trouble kids” during the school day. For example, the 
teacher described a student who initially did not pay attention in science classes; however, 
because of his participation in afterschool science activities, he has become more interested in 
his school work. She described an activity during which students learned about the life cycle of 
worms: “Yesterday [...] he got so excited to get in the soil and work with the soil. I said, ‘Let’s 
look for things, and the excitement and his behavior [...] all of a sudden, you see him show 
interest and care.’” She attributed the student’s engagement to the participant-centered focus of 
the afterschool activities. The teacher explained, “We’re feeding into their interest; that’s where 
we will get them.” 


Alignment with Schools 
Fostering relationships with school faculty 


The program benefits from a strong relationship with its host school. The provider 
organization has served the borough for more than 30 years, and it has long-standing 
relationships with several schools, many of which pre-date the DYCD OST initiative. 
Furthermore, the program director, who has served as a substitute teacher at the school, 
continues to be a presence during the school day. 


The program’s leadership makes a concerted effort to engage the school’s teachers and 
administrators in supporting program activities. One staff member said, “The idea is all of these 
enrichment programs are supposed to be a safe haven first and foremost, but we're supposed to 
be elevating some degree of skills. So rather than us do it in isolation [...] we need to work with 
[the school] to figure out what [the school needs] and how we can accomplish it.” At this 
program, this collaborative philosophy led to the creation of the school’s outdoor garden. The 
program director said that, at a meeting with the school’s principal, he asked if there were any 
specific activities in which she wanted program participants to engage. The principal requested 
that the program use the school’s outdoor terrain to create a community garden, which could also 
serve as a teaching device as part of the program’s focus on environmental science. 


In general, the school’s faculty has been supportive of the STEM initiative, and the 
program’s existing relationship with the host school brought the school’s science teacher to the 
OST program. In a meeting with one of the school’s assistant principals, the program director 
said that the program needed a staff member who could support the STEM initiative. He said, 
“Two seconds later, [the assistant principal] got on the phone and called the science teacher. 
[...] We met, talked for 10 minutes, and, boom, she was hired.” According to program staff, 
working directly with the school’s administration yielded results more quickly than if they had 
posted a job opening. It not only demonstrated that administrators had bought into the 
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program, but also sent a message to the teachers about the program’s value to the school. 
Furthermore, according to the program’s leadership, hiring from within the school has been a 
powerful public relations tool that has helped the program gain recognition in the school; as a 
result, program and school-day staff have developed partnerships that bolster both afterschool 
and school-day learning. 


The program’s leadership and the science teacher make considerable efforts to 
differentiate afterschool and in-school instruction. A provider-level staff member said, “When I 
talk to a principal about coming on board with one of our programs, or if I’m speaking directly to 
the program staff, I always say to them, ‘I want you to look at the afterschool as your opportunity 
to do all of those enrichment activities that are on the side of your textbook. [...] So the 
principals understand and the teachers understand that this is their opportunity to do the 
enrichment — to bring the lesson to life.” The science teacher said that what separates her 
in-school instruction from her afterschool instruction is the emphasis on learning concepts for 
standardized tests that occurs during the school day. In her afterschool instruction, activities are 
more hands-on and participant-centered, which the teacher believes is more effective at helping 
participants understand and remember scientific concepts. 


The program staff is confident that the program’s relationship with the host school has 
had a positive effect on participant learning. The science teacher stated that having additional 
time to explore science content is the greatest benefit of the relationship between the afterschool 
program and the school. She said, “It brings [science lessons] one step further in a way we 
cannot do ina classroom.” By speaking to the school-day teachers and administrators, the 
program’s staff has provided programming that is more academically rigorous than it has been 
in previous years. Aligning program content with the school’s student learning objectives helps 
reinforce concepts and support learning. In turn, the program staff speculate that this will 
improve participants’ self-efficacy as it relates to science: “When they're getting the 
reinforcement and the repetition and the enrichment, that they're becoming well-versed, more 
comfortable as students, more likely to pursue a higher education.” 


STEM Staff 
Hiring staff with STEM and youth development expertise 


A science teacher from the host school joined the OST program staff early in 2013 and is 
supported by the provider organization’s education specialist, an experienced museum educator 
with a passion for environmental science. In the words of a staff member from the provider 
organization, “STEM can’t be successful unless you have someone who’s well-versed or 
enthusiastic about it. There is no comparison when you have someone who enjoys that field [...] 
and is able to be responsive to the children.” 
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The program has hired and compensated 
licensed teachers using the funding received from 


Approach to staffing 


DYCD, which is possible because the program day Total number of 

is shorter than in a typical OST program. Currently, an 

the program runs two hours a day, four days each Education 

week. The science teacher was approached by the specialist supports 

school’s assistant principal, who serves as a liaison STEM 

between the program and the school. In hiring a Number of staff 

science teacher from within the school, the program who lead STEM 

secured a staff member who knows participants from aclivines 

the school and who works well with youth. The Professional ieee ea 
science teacher’s relationships with participants have development 


supported participant-centered content delivery, a 


since the science teacher knows where their interests 
lie and incorporates these interests into afterschool 
program content. Many of the participants who 
engage in STEM afterschool activities are students 


who attend the science teacher’s school-day classes. Salary range for $35/hour 
STEM staff 


Grow NYC, Compost 
NYC, NYC Parks 
Department, Friends of 
Blue Heron Park 


External partners 
supporting STEM 


To plan activities for the afterschool session, Staff paid for: 
the science teacher meets with participants to discuss 


. ” : Meetings 
which areas of science they would like to explore. 
eres ; . : Professional development 
Often, participants will describe experiments that ie 
they completed in class and pose questions about the BesSOH Pan ngane me 
design, which the teacher integrates into her Special resources STEM-specific 
afterschool lessons. One recent example of this needed for STEM curricula, kits, and 


occurred during a school-day lesson during which programming supplies 
students learned about the life cycle of worms: at the 
end of the lesson, students had additional questions about worm habitats and life cycles, so the 
science teacher created an activity for the afterschool program in which the participants would 
build a habitat for the worms and observe their physical development. 


Similarly, the provider organization hired an education specialist with experience 
working with youth and a background in environmental science. A provider-level staff member 
said, “It [was] imperative to find personnel who are passionate and knowledgeable about 
sciences and applicable mathematical concepts. [...] This was a significant consideration in the 
recruitment of personnel.” The education specialist supports staff at multiple programs as staff 
implement STEM-focused content. At this program, he and the science teacher have developed 
a symbiotic, information-sharing relationship wherein he provides her materials related to 
environmental science — a focus of the program at the school — while she provides school-day 
materials and lesson plans to use at the provider organization’s other programs. The education 
specialist has also brought in several external partners to work with the program, many of whom 
are from his existing network of STEM-related organizations. 
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Establishing external partnerships 


The program benefits from several partnerships with community organizations that 
support STEM programming, including Grow NYC, Compost NYC, the New York City Parks 
Department, and Friends of Blue Heron Park. Many of these partnerships were formed through 
the educational specialist’s existing network, and he has engaged them to provide enrichment 
activities for participants. The educational specialist suggested that STEM programming across 
OST programs would benefit if DYCD and other provider organizations were also to establish 
institutional relationships that could be used across programs. He said that while he was already 
connected with the science-related nonprofit network prior to starting this position, he recognizes 
that others might struggle in connecting with potential partner organizations. 


Staff members from these partner organizations have provided resources to support 
programming and, in some cases, led activities for participants. For example, Compost NYC has 
led composting workshops for participants, and rangers from the New York City Parks 
Department have hosted nature hikes and workshops for participants. The education specialist 
said that he is very active in the planning and delivery process when external partners work with 
the program: “TI let them do their program, but I like to have my hands in the development. I’d 
want it to be tailored toward the goals that we have set here at [the school].”» Most of the 
opportunities are provided free-of-charge to the program; when funding is required, the program 
director uses funds from his discretionary budget to cover costs. 


The executive director explained that many of these relationships are new for the 
provider organization. “[The provider organization] has been around for more than 30 years, and 
we have linkages with many organizations [in the community] that provide activities. [...] 
We’re dealing more with agencies that have a mission but don’t have a way to reach youth. 
We’ve opened our door to them and [...] it’s opened a broad horizon for additional resources.” 
The program staff explained that, for the most part, they have had success in engaging 
community organizations but would like to engage a few additional partners, including the 
Museum of Natural History and the Bronx Zoo, which they would like to engage. 
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Continuous Quality Improvement (CQI) 


Approach to CQI 


Providing feedback to experienced staff members 
Staff submit 
The DYCD Program Quality Monitoring Tool and paler deg 

the NYSAN Program Quality Assessment Tool are used 
across all the nonprofit provider organization’s programs to 


formally evaluate programs. Formal evaluations occur twice} —_ Frequency of Weekly 
per year; the program director, education specialist, and staff meetings 

provider-level staff are responsible for evaluation activities. NYSAN Program 
Broadly, the evaluations focus on program capacity to Program Quality 
provide engaging activities that help participants develop assessment Assessment, Staff 
academically and socially. There is also a focus on whether meee eee ae ere 
staff support positive youth development. For STEM, the reflection 


education specialist works with staff across all programs to 

help activity facilitators reflect on their activities and 

determine whether staff and participants are comfortable sides heed ot Twice per year 

with the material. The program director is also exploring 

STEM-specific tools to supplement the evaluation tools ; MGASUIS PIOgiess 
se of of participants 


currently in use. evaluation outcomes, Staff 
feedback development 


Staff receive ongoing feedback from the education 
specialist and the program director. The education specialist said, “I think [the provider] is 
blessed with really engaging staff. [...] They all seem to take what I say and use it, implement it, 
and grow. I think that’s the idea.” As noted above, the provider places a priority on hiring 
licensed teachers to lead program activities. The program’s leadership believes that, despite 
these teachers’ qualifications and experience, giving them feedback has not been a challenge 
because the feedback emphasizes youth development practices rather than instructional 
technique. A provider representative said, “Some of the things we’re going to ask of [teachers] 
is vastly different than what [the principal] is going to ask.” 
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On the Block Youth Program 


Program Context 


Years in operation 10 

Grades served K-8 

Center- or school-based program Center-based 

Program operation Monday-Friday, 2:30 PM — 6:00 PM 

Number of participants 160 (100 elementary-grades; 60 middle-grades) 
Number of participants in STEM 160 

STEM enrollment By grade/age-group 


STEM activity frequency/duration One hour per week (rotating schedule) 


Overview 


This center-based OST program serves approximately 160 elementary and middle 
school-aged participants who reside in the neighborhood near the center. (It is funded by two 
separate DYCD contracts, one for elementary-grades and another for middle-grades 
participants.) The program, which has operated for 10 years, provides a mix of academic and 
enrichment activities. Math and science were previously incorporated into programming, but in 
response to the new expectations for OST programs, the provider organization has begun an 
agency-wide STEM initiative to bring science-related content to all its programs. By focusing 
on STEM, the provider organization aims to bring more deliberate structure to all its programs 
and help participants develop 21" century skills. 


In line with the provider organization’s priority, this program established two goals for its 
science activities: to expose participants to new content areas and to encourage participants to 
pursue higher education and careers in STEM-related fields. In the program director’s words, 
the science initiative aims to “spark an interest in science and help [participants] build their 
knowledge.” The program director and group leaders hope to expose participants to many of the 
educational and career opportunities available to them through science activities. This goal is a 
priority, especially for female participants. During the summer program, for example, the 
program director will invite as guest speakers women who work in STEM-related industries; 
moreover, the program director plans to focus on how female scientists are recruited as a 
program strategy to attract more female participants. 


The combined elementary-grades and middle-grades program is led by one program 
director and is staffed by 14 group leaders who are responsible for implementing core activities 
and enrichment clubs. Currently, a high school graduate and a college graduate lead science 
activities for both elementary- and middle-grades students. Participants are divided by grade 
level into groups and engage in one-hour blocks of activities and clubs, including art, dance, 
technology, nutrition, cooking, and sports. While participants may choose their Friday club 
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activity, all other program activities are mandatory. Each week the program director creates a 
schedule for each grade group. Science group leaders aim to make activities “‘as enjoyable as 
possible” to keep participants engaged in each content area. To do so, most activities include a 
hands-on learning activity that differentiates the OST program instruction from the participants’ 
school instruction. 


At this program, participants engage in science activities each Thursday. Science 
activities address topics that are primarily drawn from published curricula, including 
After-School Science PLUS and Wonderwise. The program director also encourages the science 
group leaders to reference the Harcourt science curriculum, which is used by several of the 
feeder schools that the participants attend. In general, the curriculum is used as a supplement to 
activities; the program director explained that the science group leaders “are creative and use a 
variety of tools to boost the kids’ interests.” 


Approaches to STEM Programming 
Program Structure 
Nonprofit provider organization emphasis on STEM in youth programs 


The provider organization that oversees the program began a STEM initiative to promote 
STEM-related content in all its programming. In doing so, the provider organization seeks to 
enhance programming and help participants develop skills that will support their success in 
college and careers. The program director commented that the STEM initiative has made the 
program more structured; the addition of science activities, according to the program staff, has 
created an opportunity for students to develop and practice the necessary skills required to excel 
in education and careers. One group leader said, “A long time ago, we didn’t even do science in 
afterschool — not at all. [...] I think it makes the program better. It enhances it because we added 
on something that the kids need and could use in life.” 


While the provider organization has offered some guidance on STEM implementation, 
each program was given flexibility in designing and planning its science activities. Greater 
flexibility allows program leaders and staff to determine the subjects and instructional strategies 
that work best for the youth in their programs. All the program directors, for example, attend the 
same trainings provided either by the agency or by DYCD-affiliated technical assistance 
providers; however, they do not necessarily have to implement all the activities to which they are 
exposed in the trainings. The program director said, “We go to trainings and we see what works 
for our kids. Some programs do hydroponics. We don’t do that here because [...] it wouldn’t 
work with these kids. It would be good for another site. You have to know who your kids are 
and how to address them to see what works.” 
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STEM Curricula and Pedagogy 


Youth-centered content 
As part of the flexibility given to programs to pppreseh io stem euneue 
implement the STEM initiative, the provider STEM topic area(s) 
organization identified several research-based science | addressed 
curricula to be included in a menu of options from ; Atiek School Baisnes 
which programs can select. This program draws its Curdediaused PLUS, Wonderwise, 
science activities from the After-School Science Fruit Bowl 
PLUS and Wonderwise curricula. In addition, group 
leaders are encouraged to include concepts that are 
covered by the schools attended by participants. 
Besides having access to the Harcourt 

curriculum, which is used by the program’s feeder schools, the program director shares 
information gleaned from conversations with school-day teachers during weekly staff meetings. 


General science 


Provider organization, 
Online resources; 
Published curricula 


Sources for activity 
plans 


The program director and science group leaders aim to increase participant engagement 
by addressing their interests. As one group leader explained, “I think it’s easier if you ask the 
kids what they would like to learn during science.” The program director and group leader try to 
get to know all the participants in the program, including whether the content of the activities is 
something to which participants can relate and is age-appropriate. Often, one of the group 
leaders said, participants complain that they have already learned about a topic in school; in 
response, group leaders may adjust activities to supplement what was already learned in school 
or create new lessons based on participants’ interests. When working on a lesson about plants, 
for example, younger participants demonstrated basic knowledge about pollination, which they 
had learned in school. However, when participants had more questions about pollination, the 
group leader found additional resources, including a film, to answer questions and introduce new 
information. In the program director’s words, the efforts to center the activities on participants’ 
interests have improved youth engagement: “You could have a wonderful lesson, but, if the kids 
aren’t interested in it, you’re talking to yourself. We definitely try to do a lot of things 
participant-centered so we can get their feedback.” 


Teaching through hands-on, project-based learning 


The science group leaders incorporate hands-on learning experiences and visual aids into 
their lessons, both of which are intentional strategies used by the program to differentiate 
afterschool and in-school instruction. One science group leader said, “I try to make [the science 
activity] as welcoming as possible so they come to science and say, “I want to be in this room.’ 
The science group leaders explained that they try to distinguish in-school instruction from 
afterschool activities by incorporating educational games to reinforce the lessons participants 
learned in school. For example, one science group leader uses Jeopardy to teach science-related 
vocabulary. Project-based learning is also used to engage participants, and sometimes 
competition is incorporated into the projects to keep youth engaged. When the evaluation team 
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visited the program, older participants were learning how to create simple closed and open 
circuits. The participants, in groups, will use this knowledge over the next few weeks to design, 
build, and light model homes to be judged by program staff. 


To make the science activities more “their own,” the science group leaders are given 
flexibility when planning activities, and they often incorporate their own passions into the 
science activities. One group leader, for example, who described himself as an “artsy” person, 
often incorporates visual components into activities and projects. For a plant anatomy activity, 
the group leader planned for the participants to build models of plants to reinforce what they had 
learned. 


STEM Staff 
Training staff to implement STEM 


When asked what she looks for when hiring staff, the program director said that an ability 
to work with youth is a must: “I don’t care if you have a Master’s in science, if you can’t have a 
conversation with or don’t like kids, then you can’t be around them.” Although neither science 
group leader has a formal STEM-related education, they both felt that their limited STEM 
educational experiences did not pose a challenge to their work. One group leader, for example, 
said that he fills in knowledge gaps by conducting independent research and by asking for 
support from the program director. 


The provider organization and program director place a priority on staff training. The 
program director also serves as the education specialists for the program, and in that role 
supports activity planning and monitors activity implementation. The science group leaders 
explained that the program director shares information about relevant trainings and ensures that 
someone from the program attends. To implement STEM, both science group leaders have 
received training from TASC and PASE. The group leaders also attended training on the 
Wonderwise curriculum, one of the published curricula used by the program, and a content- 
related training on hydroponics, which is being implemented at another program. 


Both science group leaders said that while the trainings had been useful, they struggle in 
securing supplies to conduct activities. To compensate for the absence of materials needed for 
activities, the staff adjusts activities or finds substitutes for materials to which the center does not 
have access. The program director said, “We’ll just use what we have to make the experiment 
come to life. We have to be creative.” 


Alignment with Schools 


Connecting material to schools while maintaining 
program priorities 


The program serves participants who attend one of 
10 feeder schools in the surrounding area. As an employee 
of one of the feeder schools, the program director connects 
with the science teachers at her school to discuss strategies 
to connect the afterschool program to student learning 
during school. While some efforts are made to align 
program content to school-day learning, primarily through 
the Harcourt science curriculum used by the feeder 
schools, the program director said that the science group 
leaders maintain flexibility when planning and 
implementing science activities. “We try to connect so the 
kids can achieve academically, but [the schools’] goals are 
not necessarily what ours are.” One of the feeder schools, 
for example, is a health and wellness magnet school; 
program participants took part in a nutrition activity and 
explored food groups, based on curricula provided by City 
Harvest. While this is a strong theme at the feeder school, 
the program only occasionally integrates nutrition-based 
activities. 


The science group leaders differentiate in-school 
instruction from afterschool instruction. The program 
director explained that, by virtue of the group leaders not 
having formal education training, content delivery will 


Approach to staffing 


Total number of 
staff 


Education 
specialist 
supports STEM 


Number of staff 
who lead STEM 
activities 


Professional 
development 
sources 


External partners 
supporting STEM 


Salary range for 
STEM staff 


Staff paid for: 


Meetings 


14 


Yes (program 
director) 


PASE, TASC 


City Harvest, 
New York 
Academy of 
Sciences, 
TransCanada 


$10-15/hour 


Yes 


Professional development Yes 


Lesson planning time 


Special 
resources 
needed for STEM 
programming 


Yes 


Materials 


associated with 


curricula 


always be different. “They’re not teachers. They’re not thinking of making that connection to 
statewide standards that you have to do in school. Their lessons [are delivered] in a more fun 
way.” With respect to using the Harcourt curriculum, the program director said that she 
encourages the science group leaders to use the materials as a supplement to their activities. 
“We’re not bound to any specific curriculum, but with the creativity of the instructors, they’re 


able to do a lot of activities with the students.” 
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Continuous Quality Improvement (CQI) 


eae Approach to CQl 
Providing feedback to staff PP 
Staff submit 
All group leaders receive ongoing training and scene plans for 
: z view 

support from the program director. The program’s staff 
meet weekly on Fridays to discuss activity planning, Frequency of Weekly 
challenges experienced by group leaders, and strategies to staff meetings 

: : : NYSAN Program 
lead effective sessions. Each staff member also submits Quality 
lesson plans to the program director, who then provides Assessment tools Assessment, 
them with written feedback of their plans. Lesson plans used Youth surveys 
outline activities planned for the week. While the science Frequency of a syaae 
group leaders draw the majority of the material they teach evaluation i ie 


from published curricula, they are encouraged to adjust 
the curriculum to better serve participant interest. 


Measure student 
progress, Adjust 
Use of feedback program content, 
The program director also observes activities and Make human 


resource decisions 


provides feedback to ensure that group leaders improve (provider) 


their instructional strategies. One science group leader 
said that he received feedback on how to keep his entire class engaged during activities; the 
group leader stated that this type of feedback has helped his development as an instructor: “I feel 
that I have improved a lot, especially in science. I wasn’t really focused on science when I was 
hired, but, as time went on, [...] I gradually grasped it and became interested. I think having me 
interested in [science] makes the kids more interested.” 


To support continuous quality improvement, the program director uses the NYSAN 
Program Quality Assessment Tool to assess and improve activities. Staff evaluations are done 
using an agency-developed tool. The tool examines several domains: instructional skills, 
commitment to program, practice, and relationships with youth. The program director conducts 
staff evaluations twice per year. The first evaluation occurs at the middle of the program year 
and simply checks staff to see how they are doing, and the second evaluation occurs at the end of 
the year. Staff evaluations are entered into an agency-wide database and the agency uses the 
results to determine staff compensation, promotions, and other employment decisions. 


Using participant feedback to adjust activities 


To determine the efficacy of all program activities, the program administers pre- and 
post-surveys to participants to provide insight into what they have learned during the program 
year. For STEM, participants complete the surveys in September and May; the goals of the 
survey are to assess participant knowledge and determine the extent of their interest in science. 
The program director said that she tries to make changes to the program based on the feedback 
from the student survey. She explained that she and the science group leaders pay attention to 
youth scores on knowledge-based items. When participants complete the May survey, staff 
members evaluate student progress. If participants did not improve in science-related content 
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areas, the science group leaders and program directors make new plans to strengthen the STEM 
program area for the following school year. 


In addition, the program director and science group leaders engage participants in 
informal conversations to gain their input on science activities, particularly what they liked about 
a given activity, what they did not like, and what other types of science-related activities they 
would like to try. Next year, an evaluation specialist from the provider organization will begin 
participant focus groups; these focus group data will be used to improve the STEM initiative 
across all of the agency’s programs. 


STEM Majors 


Program Context 


Years in operation 1 

Grades served 4-8 

Center- or school-based program Center-based 

Program operation Monday-Friday, 3:00 PM- 6:00 PM 
Number of participants 170 


Number of participants in STEM All sixth- and eighth-graders participate in STEM enrichment; 
25 participants in grades 4-8 chose to “major” in STEM 


STEM enrollment Required and by interest 


Required: twice per week for 30 minutes; 


STEM activity Tequency duration By interest: three times a week for 1 hour 


Overview 


This center-based program opened during the 2012-13 school year and serves participants 
from about 16 schools. The program implements the nonprofit provider organization’s 
afterschool model, which the program director created about five years ago and which is also 
incorporated in the organization’s other programs. According to the program director, the goals 
of the model are to introduce participants to various options in school and in life, to give them 
choices in the activities they do, and to support and encourage participants as they explore these 
options. Essential elements of the model include individual coaching and enrichment activities, 
with a focus on performing arts and on STEM. 


An integral part of the program is the major/minor model. At the beginning of each 
semester, participants in fourth grade and above select a major and a minor. (They can select 
different majors and minors each semester.) For their major, participants engage in three hours 
of programming per week, choosing STEM activities or others, such as creative writing, drama, 
dance, and art. Participants also choose a minor, in which they participate for two hours a week 
choosing either newspaper, robotics, athletics, cooking class, dance, or vocals. 


Each major is led by an activity specialist with expertise in that subject area. For 
example, the STEM major and the robotics minor were both led by the STEM specialist, who is 
a pre-med college student majoring in biology. 


Approaches to STEM Programming 


STEM Curricula and Instructional Strategies 


Approach to STEM Curricula 


Differentiating STEM activities by participant 


interest STEM topic Chemistry, Biology, 
areas(s) addressed Earth science, 


Anat , Physiol 
In this program, STEM is a priority for all MERON rely steed 


participants — both those who explicitly express 

interest in it, and those who do not. All middle-grades 
participants in the program, regardless of their chosen 
major, participate in two 30-minute sessions of STEM 
activities per week. For participants with a STEM Curricula source(s) Online resources, 
major, this time is in addition to the three hours of Self-developed 
major programming each week (one hour a day, three CUnneIE 
days per week). 


Curricula used Activities developed 
by staff 


For the participants who declare an interest in STEM by majoring in it, the program seeks 
to “keep them interested in science” and gives them the opportunity to “explore different areas of 
science” such as advanced chemistry or dissection, according to the STEM specialist. She also 
hopes to get them to recognize the science that exists in everyday life. Some examples of topics 
that the STEM major group (called G-STEM) has covered include: advanced chemistry, 
dissection, cardiology, earth science, and ecology. According to the program schedule, the group 
spends about one week on each topic. For example, in the chemistry week, on the first day 
participants learned the difference between acids and bases (through a lecture/discussion led by the 
STEM activity specialist). On the second day, participants completed an activity to differentiate 
between acids and bases, and on the third and final day (the typical lab/activity day), they learned 
how to make pH sticks and how to balance out an acid. 


In addition to selecting a major, participants in the program select a minor, which they 
participate in twice a week for one hour. For example, for the first semester, participants could 
minor in robotics: the STEM activity specialist attended a robotics training to learn how to lead the 
activities. During the second semester the program intentionally did not offer robotics, in order to 
expose participants to a greater variety of activities through their minors. to be exposed to a 
variety of activities in their minors. 


The goals for the STEM enrichment class, which is mandatory for all participants, are to 
instruct participants so that they can do well in their science classes in school and in the state 
science tests. This STEM enrichment class consists of two 30-minute sessions each week. 
Participants receive a basic level of exposure to many different science topics that are aligned 
with the state science standards. The STEM enrichment class, which is taught by the STEM 
specialist, has covered such topics such as the respiratory system, environmental science, and the 
laws of energy and motion. 


Participant input and feedback play a role in the selection of activity topics. For STEM 
enrichment, the STEM specialist determines if participants need help with science as she helps 


E-2 


them with their homework during the program’s daily homework help period. That way, any 
science topic with which participants are struggling will be addressed in a future STEM 
enrichment class. For G-STEM, she talks to participants about their interests and, because she 
has had some of the same participants in G-STEM for a few semesters, she knows them well and 
knows what they are interested in. 


Structuring STEM activities in order to engage participants 


The STEM enrichment class period is limited to one half-hour and the STEM specialist 
has struggled to come up with engaging activities that will fit into the time frame. Although a 
longer time slot might help the STEM specialist go deeper into an activity and even incorporate 
some project-based and/or hands-on learning, both the project director and the STEM specialist 
said it would be harder to keep participants’ attention for an hour’s worth of STEM enrichment. 
The project director also mentioned that for middle school participants who are not explicitly 
interested in STEM, attending an hour-long “class” on STEM would be a hard sell, and those 
participants might instead skip the program altogether on that day. The class is therefore kept to 
a more manageable 30-minute “power lesson” instead of a longer session. 


The former education specialist for this program (a science teacher) discouraged the 
STEM specialist from doing hands-on activities with STEM enrichment participants in 
30-minute blocks, saying she would not have time nor space to do these types of activities with 
such a large group. The STEM specialist also said that the short time frame was not the only 
thing keeping her from doing hands-on activities with the STEM enrichment group; other factors 
included the lack of supplies and the logistical difficulty of conducting a large-group lab. The 
class has therefore turned into more of a lecture-style activity which the STEM specialist 
reported was “hard to make it not seem like school.” 


However, the new education specialist (who joined the program in the spring) said that he 
would like to reshape the structure of the STEM enrichment class so that it is possible to 
introduce hands-on activities; in fact, he thinks it is essential to do so in order to keep 
participants’ attention. His goal is to reorganize the class into a workshop structure, much the 
same as in many school-day classes. The class would begin with a five-minute “do now” 
activity that would ideally build on the lesson learned in the previous session. Then, the STEM 
specialist would introduce the topic of the day in no more than 10 minutes, condensing the topic 
into two or three essential points. In the next 10 minutes, students would complete a hands-on 
activity, followed by five minutes of wrap-up (to be resumed in the next class). The education 
specialist described creating a plan for such a class: “What's the point you want to make in that 
20 minutes? How can we do this? Also, managing classroom behavior; I think when you get 
into that workshop model when the students come in and there's a question on the board that they 
have to answer right away, that puts them in the focus right there and you have them. And then, 
I constantly rove [around the room]. [...] I want to teach [the STEM specialist] a lot of the 
teaching techniques that I use as far as to get away from being in front of the class.” 


The education specialist would like to focus future lessons on fewer topics, such as 
anatomy and physiology, in which participants have expressed an interest. The STEM specialist 
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is also well versed in these topics, as she is a pre-med biology major in college and also an EMT. 
Because of strong participant interest and the STEM specialist’s corresponding interest and 
expertise, the education specialist suggested that the STEM enrichment class and G-STEM 
activities could spend more time on these topics and use them as a starting point to explore other 
areas of science. He said: “If you're able to take a topical approach like anatomy and physiology, 
you can do one or two points a day and then you could take on a chemistry aspect, a physics 
aspect, a biology aspect, whatever it takes to fluctuate back and forth. And then I want to 
introduce careers. What are these jobs that this can lead to? And I think that's the biggest thing 
lacking in science.” 


In addition, the education specialist noted that STEM labs do not necessarily require a 
high level of resources. As a science teacher, he often has to create labs for the many students he 
has throughout the day and is experienced in designing labs that use inexpensive materials. 


Creating STEM activity plans 


The STEM specialist was leery of adopting an entire curriculum wholesale because she 
thought it would not be adapted to participants’ interests and would be too much like school. 
While the STEM specialist did not use any outside curricula, she sometimes incorporated lesson 
plans that she found online. For both G-STEM and STEM enrichment class, the STEM 
specialist drew on the websites Biology for Kids and Chemistry for Kids. She liked these sites 
because she felt that they used child-appropriate language to explain science concepts. They 
helped her translate her own science knowledge into a format that was easier to teach to her 
fourth- through eighth-grade participants. She also searched the internet for sample labs and 
activities for the topics she had chosen. She spent a considerable amount of time each week 
planning lessons from scratch and tended toward lecture-style instruction for the STEM 
enrichment class. She knew the material and had read online how to adapt the vocabulary for 
middle-school participants, so she could deliver lectures to participants based on her own prior 
knowledge. She then created or found labs to incorporate into the activity for G-STEM. She 
also created her own quizzes for STEM enrichment classes, based on the information that they 
covered in class. 


Comments from the STEM specialist highlighted the need to introduce staff to the myriad 
existing resources that are available to support STEM instruction in an afterschool setting. The 
STEM specialist valued the resources she received from the program’s new education specialist 
at the end of the year to enhance her lessons. An awareness of these existing resources and 
activity ideas may have saved the STEM specialist time at the beginning of the year. 


Incorporating participants’ interests and needs is important to the STEM specialist. 
Before designing activities, she find outs what participants need and what they are interested in, 
“because it keeps them interested and it makes them feel more important. It makes them feel 
they have some sense of control rather than being controlled, rather than being told, sit here, 
learn this, write that. [...] So that's how I feel it's better to know the kids that you have and the 
group that you have, what they're learning and what they need help in.” 
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Alignment with School 


To ensure that the content of the STEM enrichment activities is aligned with the New 
York State standards, the STEM specialist said that she spent the beginning of the year 
researching what participants should be learning between sixth and eighth grade. Since the 
program’s goal in the STEM enrichment activity has been to introduce and revisit topics from 
participants’ school-day science classes and prepare them for the state science tests, the STEM 
specialist briefly discussed these topics in an engaging way in the time allotted to the activity. 


At the beginning of each semester, all the program’s activity specialists (including the 
STEM specialist) submit a list of the daily activity plans to the director and the education 
specialist. While the G-STEM activity is not intentionally connected to school topics, it does 
cover a broad range of STEM topics, and delves more deeply into topics in which G-STEM 
participants have expressed interest. The STEM specialist described how the former education 
specialist gave her feedback on her STEM enrichment activities, which she had originally drafted 
as a shorter version of her G-STEM activity plans. The education specialist explained to her that 
the STEM enrichment class “is supposed to help [participants] connect what they’re learning in 
school right now, what’s going to help them right now or maybe in the near future, but nothing 
too advanced.” So she revised her daily activity plans for the enrichment class to connect with 
the school topics. 


All program staff agreed that hands-on activities are an important element of OST STEM 
programming, and that the STEM specialist had successfully integrated them into her smaller 
STEM major class. The education specialist believed that it was possible—and essential—to also 
do so in the shorter STEM enrichment class, and that such activities went to the heart of what 
STEM programming in afterschool could accomplish: 


“[STEM in afterschool] goes back to variety of experience and I think a lot of times 
students are taught to the test as far as the fourth-grade test and also the eighth-grade 
test. So I think [this program] can add something so special to it by actually making it 
about the kids, about the labs, about the hands-on activities as opposed to just content 
that doesn't touch upon the heart of science and doesn't touch upon inquiry. And I think 
it can add to that level and do things that are not done in school.” 


To the education specialist, STEM in OST represents an opportunity to go in-depth on science 
topics and provide hands-on opportunities in a way that school lessons might not be able to do. 
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STEM Staff 


Approach to staffing 
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majoring in biology. According to the program’s education Education 
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activity took place during the fall semester. STEM staff 


The STEM specialist said that the program conducts a sien peleiion: 


staff training in-house at the beginning of the year, “‘and then Meetings Yes 
sometimes if [the director] feels that it’s something that we Professional development No 
may need to touch base on or something we need to be Lesson planning time No 
reminded of, he’ll have a mini-training in the beginning of Special 


resources 
needed for STEM 
programming Lab materials 


the day before we start the program.” 


Continuous Quality Improvement (CQI) 
STEM activity planning 


Before the semester begins, the STEM specialist comes up with a list of topics that she 
will teach each week. The STEM specialist arrives at the program early, about noon or | p.m., 
and prepares for the day until study hall begins at 3:30 p.m. The STEM specialist’s lesson 
planning time is not paid. She noted that she spends more time planning this year than she did 
last year, even though this year is her second year teaching the program: while last year’s topics 
were somewhat disconnected, this year she is trying to connect topics with one another, which 
requires more advanced planning. 


The STEM specialist does change the planned activities if participants have not yet 
mastered a topic: “If they still don’t understand what I taught them, I’m not going to switch over. 
So it’s about staying committed to whatever you’re teaching them until they understand and then 
also keeping it interesting at the same time.” The STEM specialist checks for participant 
understanding through periodic quizzes and reviews, as well as through frequent discussion 
throughout the activities. 


Communication between OST staff to encourage program improvement 


The quality of program activities is checked through 
daily emails that staff are required to send to the director and 
by the director’s frequent, informal program observations. 
All staff must send the daily email reports to the director eieakir 
detailing what happened that day in the activities they led, lesson plans 
their challenges, and any suggestions they have. The STEM for review 
specialist described writing to the director about classroom 
management challenges she was having in the STEM 


Approach to CQI 
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enrichment course; he gave her strategies to improve her staff meetings Daily email reports 
skills. The director said he responds to these staff emails 
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of the program.) The director reported that the emails help tools used analysis 


him keep track of everything that happened at the program. 
Frequency of 


evaluation monty 


The director visits all the activities at the center every 
day for at least five minutes. He usually refrains from giving 
feedback during these visits unless he sees something out of Use of | 
the ordinary. Every month, the director sits down with each Pleats Staff development 
staff member and gives them feedback based on what he has 
seen in their activities over the past month, related to both 
instructional strategies and to content. He explained, “Depending on the staff member, I may give 
them specific things that I want to see. [For example,] in your next curriculum, I want to see more 
diversity in the genre of music that you use for dance.” Since the program ends each semester with 
a performance, the director and the staff member also assess how far along participants are in 
preparing for the performance. The STEM activity specialist said that she frequently gets feedback 
from the director “because he’s always in the class, and he sees how much my curricula have 
changed and [how] my teaching styles have changed, [from] when I first started working with him.’ 


> 


At the end of the year, the program planned to survey participants, parents, and such 
stakeholders as principals and community members in order to gauge the success of various 
aspects of the program. The director reported also planned to analyze participant attendance at 
the program to determine “what were the best things we did, [...] which majors were attended 
the most, which were attended least, which days were attended the most.” 


Finally, the program holds a weekly hour-long paid staff meeting, in which staff discuss 
and attempt to resolve any problems in the program as they arise. The director leads the 
meetings with a rundown of the past week, using the acronym WWDR for “what we did right” to 
describe something a staff person did well during the previous week. The staff members then 
share any concerns that they may have. The STEM specialist said that if something is not 
working well in the program, staff work together to devise solutions: “We look at how to 
adequately make those changes that will suit all of us that work here and also the students that 
we serve. So it's a trial-and-error type of thing. We learn. We try things. If it doesn't work, we 
look for solutions.” 
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